[R] Memory problem

[Prof Brian Ripley]

I know what the problem is: it will be fixed in 1.4.0.  There's a slow
memory leak in the internals of writeBin, and this is exercising it
very hard.

B


On Sat, 8 Dec 2001, Manfred W. Baumstark wrote:

> Uwe Ligges asked me to produce a more simple example. Ok, the problem is as
> follows: If I write big files memory utilization grows constantly. Here it
> is a little test program, and it's output:
>
> imv <- integer(length = 3000000)
>
> memory.profile()
> round(memory.size()/1048576.0, 2)
> for (j in 1:20) {
>
>   imout <- file(paste("test", j, sep=""), "wb")
>   writeBin(1:2, imout, size=2, endian="big")
>   writeBin(imv, imout, size=2, endian="big") # this one makes the problem
>   close(imout)
> }
>  memory.profile()
>  round(memory.size()/1048576.0, 2)
>
> Manfred
>
> ############ Write 20 big files
> > imv <- integer(length = 3000000)
> >
> > memory.profile()
>     NILSXP     SYMSXP    LISTSXP     CLOSXP     ENVSXP    PROMSXP
> LANGSXP
>          1       4080     104244       1280          4          0
> 53027
> SPECIALSXP BUILTINSXP    CHARSXP     LGLSXP
> INTSXP
>         59        483      10886       1510          0          0
> 24
>    REALSXP    CPLXSXP     STRSXP     DOTSXP     ANYSXP     VECSXP
> EXPRSXP
>       6693          4       6581          0          0          8
> 0
>             EXTPTRSXP
>          0          0
> > round(memory.size()/1048576.0, 2)
> [1] 20
> > for (j in 1:20) {
> +
> +   imout <- file(paste("test", j, sep=""), "wb")
> +   writeBin(1:2, imout, size=2, endian="big")
> +   writeBin(imv, imout, size=2, endian="big") # this one makes the problem
> +   close(imout)
> + }
> >  memory.profile()
>     NILSXP     SYMSXP    LISTSXP     CLOSXP     ENVSXP    PROMSXP
> LANGSXP
>          1       4081     104262       1280          4          0
> 53027
> SPECIALSXP BUILTINSXP    CHARSXP     LGLSXP
> INTSXP
>         59        483      10929       1510          0          0
> 26
>    REALSXP    CPLXSXP     STRSXP     DOTSXP     ANYSXP     VECSXP
> EXPRSXP
>       6693          4       6583          0          0          8
> 0
>             EXTPTRSXP
>          0          0
> >  round(memory.size()/1048576.0, 2)
> [1] 134.44
> >
>
> ############ Write 20 small files
> > imv <- integer(length = 3000000)
> >
> > memory.profile()
>     NILSXP     SYMSXP    LISTSXP     CLOSXP     ENVSXP    PROMSXP
> LANGSXP
>          1       4080     104244       1280          4          0
> 53027
> SPECIALSXP BUILTINSXP    CHARSXP     LGLSXP
> INTSXP
>         59        483      10886       1510          0          0
> 24
>    REALSXP    CPLXSXP     STRSXP     DOTSXP     ANYSXP     VECSXP
> EXPRSXP
>       6693          4       6581          0          0          8
> 0
>             EXTPTRSXP
>          0          0
> > round(memory.size()/1048576.0, 2)
> [1] 20
> > for (j in 1:20) {
> +
> +   imout <- file(paste("test", j, sep=""), "wb")
> +   writeBin(1:2, imout, size=2, endian="big")
> + #  writeBin(imv, imout, size=2, endian="big") # this one makes the
> problem
> +   close(imout)
> + }
> >  memory.profile()
>     NILSXP     SYMSXP    LISTSXP     CLOSXP     ENVSXP    PROMSXP
> LANGSXP
>          1       4081     104261       1280          4          0
> 53027
> SPECIALSXP BUILTINSXP    CHARSXP     LGLSXP
> INTSXP
>         59        483      10909       1510          0          0
> 26
>    REALSXP    CPLXSXP     STRSXP     DOTSXP     ANYSXP     VECSXP
> EXPRSXP
>       6693          4       6583          0          0          8
> 0
>             EXTPTRSXP
>          0          0
> >  round(memory.size()/1048576.0, 2)
> [1] 20
> >

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._