[R] Memory allocation

[Prof Brian Ripley]

On Thu, 7 Sep 2006, alex lam (RI) wrote:

> Dear list,
> 
> I have been trying to run the function "qvalue" under the package qvalue
> on a vector with about 20 million values.
> 
> > asso_p.qvalue<-qvalue(asso_p.vector)
> Error: cannot allocate vector of size 156513 Kb
> > sessionInfo()
> Version 2.3.1 (2006-06-01)
> i686-pc-linux-gnu
> 
> attached base packages:
> [1] "methods"   "stats"     "graphics"  "grDevices" "utils"
> "datasets"
> [7] "base"
> 
> other attached packages:
> qvalue
>  "1.1"
> > gc()
>             used  (Mb) gc trigger   (Mb)  max used   (Mb)
> Ncells    320188   8.6   23540643  628.7  20464901  546.5
> Vcells 101232265 772.4  294421000 2246.3 291161136 2221.4
> 
> I have been told that the linux box has 4Gb of RAM, so it should be able
> to do better than this.

But it also has a 4Gb/process address space, and of that some (1Gb?) is 
reserved for the system.  So it is quite possible that with 2.2Gb used you 
are unable to find any large blocks.

> I searched the FAQ and found some tips on increasing memory size, but
> they seem to be windows specific, such as memory.size() and the
> -max-mem-size flag. On my linux box R didn't recognise them.

?"Memory-limits" is the key

     Error messages beginning 'cannot allocate vector of size' indicate
     a failure to obtain memory, either because the size exceeded the
     address-space limit for a process or, more likely, because the
     system was unable to provide the memory.  Note that on a 32-bit OS
     there may well be enough free memory available, but not a large
     enough contiguous block of address space into which to map it.

> I don't understand the meaning of max-vsize, max-nsize and max-ppsize.
> Any help on how to increase the memory allocation on linux is much
> appreciated.

Get a 64-bit OS.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595