[R] load huge image
maechler at stat.math.ethz.ch
Mon Mar 27 09:49:56 CEST 2006
>>>>> "Gottfried" == Gottfried Gruber <ggruber at terminal.at>
>>>>> on Sun, 26 Mar 2006 10:27:35 +0200 writes:
Gottfried> hello, i have run around 65000 regressions and
Gottfried> stored them in a list. then i stored the session
Gottfried> with save.image on my hard disk. the file is
Gottfried> almost 1GB. when i now want to load the image it
Gottfried> took tons of time. even after 12h of loading it
Gottfried> was not done, although the saving was done fairly
I'm sure it takes so lang because you (i.e. R) run out of RAM
and the machine starts to swap.
Try to get access to a Linux (or other Unix-alike) machine with
a 64-bit version of R and about 8 GB of RAM (maybe 4 GB is
already sufficient). I guess then you should be able to read it
much more quickly.
For 65000 regressions, do you need more than the estimated
coefficients or -- a bit more informatively -- the
coef(summary( <lm-fit> )) result ?
If you had only saved these coefficient matrices, I'm sure you'd
have need **much** less memory.
Gottfried> i fear i have to run the regressions again and
Gottfried> store them in a database ...
or really store what you need instead of everything ...
Gottfried> can i load this file? any suggestions?
Gottfried> thanks & bets regards, gg --
Gottfried> Gottfried Gruber
Gottfried> mailto:gottfried.gruber at terminal.at www:
Gottfried> R-help at stat.math.ethz.ch mailing list
Gottfried> PLEASE do read the posting guide!
More information about the R-help