[R] Error: cannot allocate vector of size 1125522 Kb, Reached total allocation of 510Mb

[Prof Brian Ripley]

On Mon, 4 Oct 2004, Uwe Ligges wrote:

> Luis Rideau Cruz wrote:
> 
> > R-help
> > 
> > I'm trying to compute the 'dist' function of a data set consisting of
> > 16975 observations and 5 variables(2 quantitative and 3 categorical).
> 
> So what you get returned would be a 16975x16975 matrix
> that inherits 288150625 elements.
> You need 8 bytes per element, hence 8 * 288150625 = 2305205000, to 
> represent the matrix in memory (only once!, and you might need twice or 
> more available memory to work with it).

Not quite, as it only stores half.  In fact you need around 1125522Kb, 
just over 1Gb.

> So you need as a lower bound 2.5 GB of memory, and I guess 5 GB are 
> required in order to get it to work. If you have that much available, 
> try again. But since you are on Windows (and have not told us), you 
> might not have that much....

It has an address space limit of 2Gb unless worked on: see the rw-FAQ.

> > If I call the function on a subset of the data frame everything works
> > fine but when I reach above 3000 observations R either crash or gives
> > the following error message.
> > 
> > Error: cannot allocate vector of size 1125522 Kb
> > In addition: Warning messages: 
> > 1: NAs introduced by coercion 
> > 2: Reached total allocation of 510Mb: see help(memory.size) 
> > 
> > I have tried 'help(memory.size)' and increase the 'memory.limit' but
> > without success

You cannot increase the limit once you have reached it, AFAIK.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595