dealing with large objects -- memory wasting ?
Martin Maechler
Martin Maechler <maechler@stat.math.ethz.ch>
Fri, 4 Jun 1999 17:21:18 +0200
Consider the following:
> gcinfo(TRUE)
[1] FALSE
> rm(list=ls()); gc()
free total
Ncells 135837 250000
Vcells 747306 786432
------
> n <- 10000; p <- 20; X <- matrix(rnorm(n*p), n,p); gc()
Garbage collection [nr. 23]...
135839 cons cells free (54%)
4275 Kbytes of heap free (69%)
free total
Ncells 135829 250000
Vcells 547299 786432
------
which is roughly 200000 less than before;
i.e. Vcells are 8-byte sized,
1 Vcell = 1 double or 2 integers
Since we have 747 thousands of them ,
constructing X the double size (400'000) shouldn't be a problem ...
> rm(X); n <- 20000; p <- 20; X <- matrix(rnorm(n*p), n,p); gc()
Garbage collection [nr. 25]...
135823 cons cells free (54%)
2713 Kbytes of heap free (44%)
Error: heap memory (6144 Kb) exhausted [needed 3125 Kb more]
See "help(Memory)" on how to increase the heap size.
but it *is*, and it's
matrix' fault. (it constructs x, i.e. effectively doubles it's argument).
----
There seem to be worse problems when use
var(x)
and x is one of those huge n x p matrices...
--------
Of course there are problems like these to be optimized -- ``everywhere''.
Any proposals for a general approach?
Martin
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._