[R] arrays in binary files

[Duncan Murdoch]

>In the writing of functions for a package have to deal with (very)
> large arrays and I was considering that
>having them in files rather than in memory would help me a bit
>on the memory side of the things. I was thinking of accessing the
>elements in the arrays by 'seek' and friends in C. 

My gut feeling is that this is a bad thing to do.  Unless you go to a
lot of trouble to optimize, it's generally better to let the operating
system create "virtual memory" and handle the seeking and such.

The main case where this is not true is the case of large static
arrays that change little from run to run, of which you only use a
small proportion of the entries.  Unless you can do memory mapped
files, you'll have to read the whole file into virtual memory on each
run, and that's slow.  If you're accessing every element anyway, then
it's not such a penalty to read the whole thing at the start; if they
are temporary for the run, or change frequently, then you'll be
writing the whole thing every time anyway.

Duncan Murdoch
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._