[R] load ing and saving R objects

[Prof Brian Ripley]

On Tue, 14 Jun 2005, Barry Rowlingson wrote:

> Richard Mott wrote:
>> Does anyone know a way to do the following:
>>
>> Save a large number of R objects to a file (like load() does) but then
>> read back only a small named subset of them . As far as I can see,
>> load() reads back everything.
>
>  Save them to individual files when you generate them?
>
>  for(i in 1:15000){
>
>   m=generateBigMatrix(i)
>
>   filename=paste("BigMatrix-",i,".Rdata",sep='')
>   save(m,file=filename)
>  }
>
> Note that load will always overwrite 'm', so to load a sample of them in
> you'll need to do something like this:
>
>  bigSamples=list()
>
>  for(i in sample(15000,N)){
>    filename=paste("BigMatrix-",i,".Rdata",sep='')
>    load(filename)
>    bigSamples[[i]]=m
>  }
>
>  But there may be a more efficient way to string up a big list like
> that, I can never remember - get it working, then worry about optimisation.

(Yes, use bigSamples <- vector("list", 15000) first.)

>  I hope your filesystem is happy with 15000 objects in it. I would
> dedicate a folder or directory for just these objects' files, since it
> then becomes near impossible to see anything other than the big matrix
> files...

.readRDS/.saveRDS might be a better way to do this, and avoids always 
restoring to "m".

If your file system does not like 15000 files you can always save in a 
DBMS.

I did once look into restoring just some of the objects in a save()ed 
file, but it is not really possible to do so efficiently due to sharing 
between objects.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595