[R] load ing and saving R objects

[bogdan romocea]

> On Tue, 14 Jun 2005, Prof Brian Ripley wrote:
> If your file system does not like 15000 files you can always 
> save in a DBMS.

Or, switch to a better/more appropriate file system:
http://en.wikipedia.org/wiki/Comparison_of_file_systems
ReiserFS would allow you to store up to about 1.2 million files in a directory.


> -----Original Message-----
> From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk]
> Sent: Tuesday, June 14, 2005 10:41 AM
> To: Barry Rowlingson
> Cc: r-help at stat.math.ethz.ch; Richard Mott
> Subject: Re: [R] load ing and saving R objects
> 
> 
> On Tue, 14 Jun 2005, Barry Rowlingson wrote:
> 
> > Richard Mott wrote:
> >> Does anyone know a way to do the following:
> >>
> >> Save a large number of R objects to a file (like load() 
> does) but then
> >> read back only a small named subset of them . As far as I can see,
> >> load() reads back everything.
> >
> >  Save them to individual files when you generate them?
> >
> >  for(i in 1:15000){
> >
> >   m=generateBigMatrix(i)
> >
> >   filename=paste("BigMatrix-",i,".Rdata",sep='')
> >   save(m,file=filename)
> >  }
> >
> > Note that load will always overwrite 'm', so to load a 
> sample of them in
> > you'll need to do something like this:
> >
> >  bigSamples=list()
> >
> >  for(i in sample(15000,N)){
> >    filename=paste("BigMatrix-",i,".Rdata",sep='')
> >    load(filename)
> >    bigSamples[[i]]=m
> >  }
> >
> >  But there may be a more efficient way to string up a big list like
> > that, I can never remember - get it working, then worry 
> about optimisation.
> 
> (Yes, use bigSamples <- vector("list", 15000) first.)
> 
> >  I hope your filesystem is happy with 15000 objects in it. I would
> > dedicate a folder or directory for just these objects' 
> files, since it
> > then becomes near impossible to see anything other than the 
> big matrix
> > files...
> 
> .readRDS/.saveRDS might be a better way to do this, and avoids always 
> restoring to "m".
> 
> If your file system does not like 15000 files you can always 
> save in a 
> DBMS.
> 
> I did once look into restoring just some of the objects in a save()ed 
> file, but it is not really possible to do so efficiently due 
> to sharing 
> between objects.
> 
> -- 
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
>