[R] Options to extend memory limit

[Marc Schwartz]

On Oct 11, 2012, at 9:55 AM, "Sebastian P. Luque" <spluque at gmail.com> wrote:

> On Thu, 11 Oct 2012 14:45:16 +0200,
> <jennifer.moeller-gulland at de.pwc.com> wrote:
> 
>> Dear All, at the moment I am using R for calculations of large
>> databases.  Unfortunately, R only manages to complete certain
>> operations at some times, and not at others. I usually get the error
>> message "cannot allocate vector of size XX"
> 
>> I am using the 64-bit version with Windows 7. While my computer has 8
>> RAM, I do have a feeling that R cannot use all of it.
> 
>> Searching online, I found that you can increase the memory with the
>> options --max-mem-size/ --max-ppsize or change the environment
>> variable R_MAX_MEM_SIZE to "allow deep recursion or large and
>> complicated calculations to be done"
> 
>> Unfortunately, I am not very knowledgable yet on how to use R and I
>> did not quite manage to use the commands successfully. Could you
>> please tell me whether these do make sense for my case and if so how
>> (and at what stage of the process) I can use them?
> 
> Are you sure you're using the 64 bit R executable which comes with the R
> installation?

Sebastian hit on my initial thought here, though depending upon how much data you are dealing with, 8Gb may indeed not be enough and some of your RAM may be used by other processes/applications, leaving less for R.

A quick check to see which version you are running is to use:

  .Machine$sizeof.pointer

If it returns 8, you are using the 64 bit version of R. If it comes back with 4, you are using the 32 bit version of R, which of course will be more limited in how much RAM it can access.

If it returns 8, then as Ben noted, you may want to evaluate some of the Large Memory options on the HPC task view:

  http://cran.r-project.org/web/views/HighPerformanceComputing.html

or of course install more RAM.

Regards,

Marc Schwartz