[R] Once again: Error: cannot allocate vector of size

[Viechtbauer Wolfgang (STAT)]

The problem I see here is not so much the 27136 observations, but the fact that the first two factors have up to 101 different levels and the third factor up to 1001 different levels. That means that lm() is essentially creating 100+100+1000 dummies for those factors, leading to a large (and sparse) design matrix (containing over 30 million entries).

Best,

--
Wolfgang Viechtbauer                        http://www.wvbauer.com/
Department of Methodology and Statistics    Tel: +31 (43) 388-2277
School for Public Health and Primary Care   Office Location:
Maastricht University, P.O. Box 616         Room B2.01 (second floor)
6200 MD Maastricht, The Netherlands         Debyeplein 1 (Randwyck)


----Original Message----
From: r-help-bounces at r-project.org
[mailto:r-help-bounces at r-project.org] On Behalf Of Matthew Dowle Sent:
Friday, January 22, 2010 17:33 To: r-help at stat.math.ethz.ch
Subject: Re: [R] Once again: Error: cannot allocate vector of size

> Fantastic. You're much more likely to get a response now.  Best of
> luck.
>
> "werner w" <pensterfuzzer at yahoo.de> wrote in message
> news:1264175935970-1100164.post at n4.nabble.com...
>>
>> Thanks Matthew, you are absolutely right.
>>
>> I am working on Windows XP SP2 32bit with R versions 2.9.1.
>>
>> Here is an example:
>>  d <- as.data.frame(matrix(trunc(rnorm(6*27136, 10000, 100)),ncol=6))
>> d[,4:5] <- trunc(100*runif(2*27136, 0, 1))  d[,6] <-
>> trunc(1000*runif(27136, 0, 1))  for (i in 4:6) d[,i] <-
>> as.factor(d[,i])  lm(V1 ~ log(V2) + log(V3) + V4 + V5 + V6, data=d)
>>  memory.size(max=F) memory.size(max=T)
>>
>> I managed to get it run through after setting the 3GB switch for
>> Windows and
>> with a clean R session.
>> I also noticed later, that after removing na.action=na.exclude more
>> regressions run through.
>>
>> But before and after the lm() it seems there should be enough memory
>> which means that lm() builds up some quite large objects during its
>> computations? --
>> View this message in context:
>> http://n4.nabble.com/Once-again-Error-cannot-allocate-vector-of-size-tp1083506p1100164.html
>> Sent from the R help mailing list archive at Nabble.com.