[R] What is the maximum limit for an array?

[David Winsemius]

On Jul 12, 2013, at 6:42 AM, Ben Bolker wrote:

> Jonsson <amen.alyaari <at> Bordeaux.inra.fr> writes:
> 
>> 
>> Hello All,
>> I am having a problem with this:
>> 
>> file<-array(dim=c(1440,720,700,3))
>>      Error in array(dim = c(1440, 720, 700, 3)) : 
>>       'dim' specifies too large an array
>> 
>> I have a memory of 20GB, But I do not know where is the problem!Any help
>> 
>> When I replaced 700 by any number bellow like( 600,500),it worked without
>> any problem.
> 
> From
> 
> http://stat.ethz.ch/R-manual/R-devel/library/base/html/LongVectors.html :
> 
> Prior to R 3.0.0, all vectors in R were restricted to at most 2^31 - 1 
> 
>> 1440*720*700*3
> [1] 2177280000
>> 2^31-1
> [1] 2147483647
> 
>  So you need R>=3.0.0 (on a 64-bit system).

And (unfortunately) perhaps three times the currently installed RAM will be needed for productive use of such an R data-object. The memory requirements of a numeric array are roughly 10 times the product of the dimensions:

10*prod( c(1440,720,700,3))
#[1] 21772800000     
#     G  M  K

So the hardware limitations are now a constraint. I am a bit surprised that object with those lower dimensions could be handled "without any problem." Generally an object of dim =c(1440,720,700,3) will not be able to be productively used for anything except read access. Copying it or even assigning new values to it, which of necessity creates a copy,  would generally overflow installed RAM and push your session into "virtual memory" at which point my system starts to display molasses-like behavior. Occassionally waiting on the order of 5-20 minutes allows the process to complete, but in many instances terminating the session is needed.

-- 
David Winsemius
Alameda, CA, USA