[R] memory management

[William Dunlap]

You need to walk through the objects, checking for
environments on each component or attribute of an
object.  You also have to look at the parent.env
of each environment found.  E.g.,
  > f <- function(n) {
  +   d <- data.frame(y = rnorm(n), x = rnorm(n))
  +   lm(y ~ poly(x, 4), data=d)
  + }
  > z <- f(1e5)
  > environment(z)
  NULL
  > object.size(z)
  21610708 bytes
  > sapply(z, object.size)
   coefficients     residuals       effects 
            384       4400104       1200336 
           rank fitted.values        assign 
             32       4400104            56 
             qr   df.residual       xlevels 
        7601232            32           104 
           call         terms         model 
            508          2804       4004276
  > environment(z$terms)
  <environment: 0x0abb86e4>
  > eapply(environment(z$terms), object.size)
  $d
  1600448 bytes

  $n
  32 bytes

Coding this is tedious; the codetools package may make it
easier.  Summing the sizes may well give an overestimate
of the memory actually used, since several objects may
share the same memory.

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com 

> -----Original Message-----
> From: Sam Steingold [mailto:sam.steingold at gmail.com] On Behalf Of Sam Steingold
> Sent: Tuesday, February 28, 2012 2:56 PM
> To: r-help at r-project.org; William Dunlap
> Subject: Re: memory management
> 
> > * William Dunlap <jqhaync at gvopb.pbz> [2012-02-28 20:19:06 +0000]:
> >
> > Look into environments that may be stored with your data.
> 
> thanks, but I see nothing like that:
> 
> for (n in ls(all.names = TRUE)) {
>   o <- get(n)
>   print(object.size(o), units="Kb")
>   e <- environment(o)
>   if (!identical(e,NULL) && !identical(e,.GlobalEnv)) {
>     print(e)
>     print(eapply(e,object.size))
>   }
> }
> 25.8 Kb
> 0.5 Kb
> 49.1 Kb
> 0.1 Kb
> 30.8 Kb
> 13.6 Kb
> 17.4 Kb
> 59.4 Kb
> 52.2 Kb
> 0.1 Kb
> 3.9 Kb
> 49.1 Kb
> 21.2 Kb
> 0.1 Kb
> 0.1 Kb
> 51 Kb
> 13.2 Kb
> 53.5 Kb
> 18.1 Kb
> 64.3 Kb
> 25.8 Kb
> 33.5 Kb
> 0.1 Kb
> 0.1 Kb
> 8 Kb
> 10 Kb
> 15.7 Kb
> 15.6 Kb
> 9.9 Kb
> 401672.7 Kb
> 19.1 Kb
> 76 Kb
> 12 Kb
> 32.4 Kb
> 156.3 Kb
> 13.1 Kb
> 20.5 Kb
> 21.8 Kb
> 10.8 Kb
> 
> sum(unlist(lapply(lapply(ls(all.names = TRUE),get),object.size)))
> [1] 412351928
> 
> i.e., total of data is about 400MB.
> why does the process take in access of 1GB?
> 
> top: 1235m 1.1g 4452 S    0 14.6   7:12.27 R
> 
> --
> Sam Steingold (http://sds.podval.org/) on Ubuntu 11.10 (oneiric) X 11.0.11004000
> http://www.childpsy.net/ http://pmw.org.il http://camera.org
> http://dhimmi.com http://palestinefacts.org http://ffii.org
> Fighting for peace is like screwing for virginity.