[R] Memory Management under Linux: Problems to allocate large amounts of data

[Prof Brian Ripley]

On Thu, 30 Jun 2005, Dubravko Dolic wrote:

> Dear Prof. Ripley.
>
> Thank You for Your quick answer. Your right by assuming that we run R on 
> a 32bit System. My technician tried to install R on a emulated 64bit 
> Opteron machine which led into some trouble. Maybe because the Opteron 
> includes a 32bit Processor which emulates 64bit (AMD64 x86_64). As You 
> seem to have good experience with running R on a 64bit OS I feel 
> encouraged to have another try for this.

It should work out of the box on an Opteron Linux systen: it does for 
example on FC3 and SuSE 9.x.  Some earlier Linux distros for x86_64 are
not fully 64-bit, but we ran R on FC2 (although some packages could not be 
installed).

Trying to build a 32-bit version of R on FC3 does not work for me: the 
wrong libgcc_s is found.  (One might want a 32-bit version for speed on 
small tasks.)

> -----Ursprüngliche Nachricht-----
> Von: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk]
> Gesendet: Mittwoch, 29. Juni 2005 15:18
> An: Dubravko Dolic
> Cc: r-help at stat.math.ethz.ch
> Betreff: Re: [R] Memory Management under Linux: Problems to allocate large amounts of data
>
> Let's assume this is a 32-bit Xeon and a 32-bit OS (there are
> 64-bit-capable Xeons).  Then a user process like R gets a 4GB address
> space, 1GB of which is reserved for the kernel.  So R has a 3GB address
> space, and it is trying to allocate a 2GB contigous chunk.  Because of
> memory fragmentation that is quite unlikely to succeed.
>
> We run 64-bit OSes on all our machines with 2GB or more RAM, for this
> reason.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595