[R] R File IO Slow?

[Marc Schwartz]

On Thu, 2007-03-01 at 09:22 -0800, ramzi abboud wrote:
> Is R file IO slow in general or am I missing
> something?  It takes me 5 minutes to do a load(MYFILE)
> where MYFILE is a 27 MB Rdata file.  Is there any way
> to speed this up?  
> 
> The one idea I have is having R call a C or Perl
> routine, reading the file in that language, converting
> the data in to R objects, then sending them back into
> R.  This is more work that I want to do, however, in
> loading Rdata files.
> 
> Any ideas would be appreciated.
> Ramzi Aboud
> University of Rochester

Here are some timings on my system, which runs Linux on a 3.2 Ghz P4
with 2 Gb of RAM and a 7200 rpm HD. I typically get around 28 Mb/sec
throughput on this drive, which is about 15% lower than normal, as it is
an encrypted partition using 256 bit AES.


> Vec <- 1:15000000

> system.time(save(Vec, file = "Vec.RData"))
[1] 33.297  0.565 38.889  0.000  0.000

# File is ~29 Mb
> file.info("Vec.RData")$size
[1] 30112009

> system.time(load("Vec.RData"))
[1] 5.607 0.167 6.575 0.000 0.000


Not terribly burdensome...

You might want to be sure that you are not low on RAM, resulting in a
lot of swapping to disk, or perhaps just a slow drive.

HTH,

Marc Schwartz