[R] memory issues

[Prof Brian Ripley]

I think the clue is that the message you quote comes from gstat, which 
does not use R's memory allocator.  It is gstat and not R that has failed 
to allocate memory.

Try re-reading the help page for memory.size.  'max=T' does not indicate 
the limit (that is the job of memory.limit()), but the maximum 'used' 
(acquired from the OS) in that session.  So 19Mb looks reasonable for R's 
usage.

I don't understand the message from memory.limit() (and the formatting is 
odd).  memory.limit() does not call max() (it is entirely internal), so I 
wonder if that really is the output from that command.  (If you can 
reproduce it, please let us have precise reproduction instructions.)

There isn't much point in increasing the memory limit from the default 
1.5Gb on a 2Gb XP machine.  The problem is that the user address space 
limit is 2Gb and fragmentation means that you are unlikely to be able to 
get over 1.5Gb unless you have very many small objects, in which case R 
will run very slowly.  In any case, that is not the issue here.

On Wed, 16 Apr 2008, Dave Depew wrote:

> Hi all,
> I've read the R for windows FAQ and am a little confused re:
> memory.limit and memory.size
>
> to start using R 2.6.2 on WinXP, 2GB RAM, I have the command line "sdi
> --max-mem-size=2047M"
> Once the Rgui is open, memory.limit() returns 2047, memory.size()
> returns 11.315, and memory.size(max=T) returns 19.615
>
> Shouldn't memory.size(max=T) return 2047?
>
> Upon running several operations involving kriging (gstat package,
> original data file 3 variables, 12000 observations)
> the program runs out of memory
>
> "memory.c", line 57: can't allocate memory in function m_get()
> Error in predict.gstat(fit.uk, newdata = EcoSAV.grid.clip.spxdf,
> debug.level = -2,  :
>  m_get
>
> Immediately following this,
>
> memory.limit() returns [1] -Inf
>                        Warning message:
>                            In memory.limit() : no non-missing arguments
> to max; returning -Inf
>
> memory.size() returns 24.573.
>
> memory.size(max=T) returns 46.75
>
> To my untrained eye, it appears that R is not being allowed access to
> the full memory limit specified in the cmd line....if this is the case,
> how does one ensure that R is getting access to the full allotment of RAM?
> Any insight is appreciated...
>
>
> > sessionInfo()
> R version 2.6.2 (2008-02-08)
> i386-pc-mingw32
>
> locale:
> LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
> States.1252;LC_MONETARY=English_United
> States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats     graphics  grDevices datasets  tcltk     utils
> methods   base
>
> other attached packages:
> [1] maptools_0.7-7 foreign_0.8-23 gstat_0.9-43   rgdal_0.5-24
> lattice_0.17-4 sp_0.9-23      svSocket_0.9-5 svIO_0.9-5
> R2HTML_1.58    svMisc_0.9-5   svIDE_0.9-5
>
> loaded via a namespace (and not attached):
> [1] grid_2.6.2  tools_2.6.2
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595