[Rd] package development

[Duncan Murdoch]

On 11/12/2008 6:04 PM, Terry Therneau wrote:
>   I'm making the move of the survival package from my own environment to,
> and have stumbled into a vacuum.   The R Extensions manual has really nice
> instructions about how to lay out the directories, order the files, and
> run tests for DISTRIBUTION of a product, but I can't find anything on how
> to set up a reasonable DEVELOPMENT environment.
>    In my local world, I had the .c and .s files in a common directory, with
> a Makefile that I had created, and the test suite in a subdirectory.  Debugging
> and development was quite nice.
> 	make
> 	cd test
> 	R
> 	attach("..")
> 		try something and perhaps it fails
> 	q()
> 	cd ..
> Fix and repeat. The Makefile took some time to create, but paid for itself a
> hundred times over. 
> 
>   So, I've now rearranged everything into standard R order.  Then I did the
> only thing I could find
> 	R CMD INSTALL ~/myRlib  survival  
> where "survival" is said directory.   This turns out to be not useful at all.
> The survival package is large, and I rather suspected that I would goof 
> something up, and I did, resulting in the following message
> 
> 	Error in parse(n = -1, file = file) : unexpected end of input at
>         14450: }
>         14451: 
> 
> It is not exactly obvious which of the 132 files in my R/ directory is the 
> culprit here.

That's something I would like to fix too.  There are (at least) two 
possible ways:  stop concatenating the files (which is not really needed 
nowadays, most packages install saved images), or add some markup to the 
concatenated file so that the parser can report on the original filename 
and line number (like the #LINE directives output by the C preprocessor).

> 
>    In general:
> 1. The library is large, and recompiling/reparsing everything is very far from
> instantaneous.  It is not the edit/load cycle I desire.

If you install from the directory, the compiling should only be done 
once (unless you change a file, of course).  (The alternative is 
installing from the tarball, which is recommended for later stages of 
testing before distribution, because it's possible something could go 
wrong in building the tarball.  But it won't include any object files, 
so you'll recompile every time.)

You can also use option "--docs=none" to skip building the help system; 
this will save a bit of time.


> 
> 2. I take testing seriously: the test suite takes on the order of 15 minutes to
> run on a fast machine.  I most certainly don't want to run it in the mid cycle.

I don't quite follow this.  If you want to run all your tests, you would 
use R CMD check.  If you only want to run some of them, then you can 
source things out of the tests directory while running interactively.

>    Someone must have tackled this.  I'm hoping that there is some documentation
> that I have managed to overlook which discussess a good setup for this middle
> ground between concieving of a library and packaging it for delivery; the
> "build, test, make sure it acually works" part of development.

I find the process I follow is to organize the files in the distribution 
structure from the beginning.   When adding new functions, I'll 
generally use source() a few times to get the syntax right, and perhaps 
run simple tests.  (But remember, if you use a NAMESPACE, the functions 
may not behave the same when they're sourced into the global 
environment.)  In the early stages, I'll do a lot of installs of the 
packages.

If I was porting a big package and wanted to find syntax errors, to work 
around the not-very-helpful error message you saw I'd do something like

for (f in list.files("pkg/R", full=TRUE)) source(f)

which will report the error more informatively.

Duncan Murdoch