[R] R File IO Slow?

[Ranjan Maitra]

I decided to run an experiment: just reading in a file which is 78MB in binary format (of ints). It takes less than 30s using a laptop with 512 MB RAM, 2.3 GHz Intel-4 single processor. At that point, I did not notice that Ramzi was talking about a .RData file.

For huge files, I usually do not save my files. I run the R code whenever I need it: the entire exercise usually takes a few minutes, at the most. If something takes very long, I usually save the output into a file and read from there. I have found that this is more efficient (besides helping in reproducing my results).

HTH!
Ranjan


On Thu, 01 Mar 2007 13:04:54 -0500 "Roger D. Peng" <rdpeng at gmail.com> wrote:

> A 27MB .RData file is relatively big, in may experience.  What do you think is 
> slow?  Maybe it's your computer that is slow?
> 
> -roger
> 
> ramzi abboud wrote:
> > Is R file IO slow in general or am I missing
> > something?  It takes me 5 minutes to do a load(MYFILE)
> > where MYFILE is a 27 MB Rdata file.  Is there any way
> > to speed this up?  
> > 
> > The one idea I have is having R call a C or Perl
> > routine, reading the file in that language, converting
> > the data in to R objects, then sending them back into
> > R.  This is more work that I want to do, however, in
> > loading Rdata files.
> > 
> > Any ideas would be appreciated.
> > Ramzi Aboud
> > University of Rochester
> > 
> > 
> > 
> > 
> > 
> >  
> > ____________________________________________________________________________________
> > Need Mail bonding?
> > 
> > ______________________________________________
> > R-help at stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> > 
> 
> -- 
> Roger D. Peng  |  http://www.biostat.jhsph.edu/~rpeng/
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>