[R] maximum matrix size if it runs with 64-bit R

[Henrik Bengtsson]

On Tue, Oct 5, 2010 at 5:21 PM, Carrie Li <carrieandstat at gmail.com> wrote:
> I am sorry that it has been couple days.
> I've read the website you provided below, but still don't quite know if this
> is doable.
> The maximum vector length is 2^31-1, so here is what I tired, and it
> returned errors as below.
>
>
> P=20000
> D=matrix(rep(0, P*P), nrow=P)
>
> Error: cannot allocate vector of size 1.5 Gb
> In addition: Warning messages:
> 1: In as.vector(data) :
>  Reached total allocation of 1535Mb: see help(memory.size)
>
> On the manuals, it says "32-bit OSes by default limit file sizes to 2GB ",
> so why P=20000 is not working here ?

Every double value (e.g. 0) needs 8 bytes.  So the total memory needed
for that matrix is 8*P*P = 3.2e+09 bytes = 3.2e+09/1024^3 Gb = 2.98Gb.
 Now, in order to do anything useful you also need space for creating
an internal copy or two of that object.  That is, you basically need
2-3 times more *free* (and *contiguous/non-fragmented*) RAM than that
to do anything useful.

/Henrik

>
> Thanks for any helps. I appreciate.
>
>
> On Sat, Oct 2, 2010 at 12:22 PM, Marc Schwartz <marc_schwartz at me.com> wrote:
>
>> On Oct 2, 2010, at 11:14 AM, Carrie Li wrote:
>>
>> > Hi everyone,
>> >
>> > If I run on a 64-bit R, what is the maximum matrix size that it can
>> handle ?
>> > Is a matrix 20,000 x 20,000 possible on 32 bit ?
>> > Thanks for answering!
>>
>>
>> A matrix is a vector with 'dim' attributes. The maximum vector length is
>> 2^31 - 1 and that does not change between 32 and 64 bit R. The primary
>> advantage of 64 bit R is the larger memory address space.
>>
>> See:
>>
>>
>> http://cran.r-project.org/doc/manuals/R-admin.html#Choosing-between-32_002d-and-64_002dbit-builds
>>
>> HTH,
>>
>> Marc Schwartz
>>
>>
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>