[R] Sweave for "big" data analysis

[Duncan Murdoch]

On 31/12/2010 3:35 PM, Lars Bishop wrote:
> Hi,
>
> Maybe I'm missing the point here...but let's suppose you are working with
> "large" data sets and using functions that take a significant amount of time
> to run in R. I woulnd't like to run these functions every time I call
> Sweave("myfile.Rnw") within R. What is the "common" practice to use Sweave
> in these situations. I would just run the function once, save the results
> and only load them each time I run Sweave on the .Rnw file. Makes sense?
>
> Sorry, the question seems silly, but I'd appreciate your thoughts.

As others have said, there are packages that provide caching.

I haven't used them, because I like to keep my projects as 
self-contained as possible:  adding a dependency on one of those 
packages is undesirable[1].  What I do in the case where there are time 
consuming calculations is to do all the calculations in a script, and 
save the results (using save()).  Then the Sweave document will load the 
objects (using load()) and do post-processing, plotting, etc.

Duncan Murdoch

1.  I do generally write things that are dependent on my own patchDVI 
package, and curse myself for the dependency all the time.