[R] glm-poisson fitting 400.000 records

[Uwe Ligges]

On 21.10.2011 23:14, Ken wrote:
> Your memory shouldn't be capped there,

Where? You cannot know from the output below.


> try ?memory.size and ?memory.limit. Background less things.
>     Good luck,
>       Ken Hutchison
>
> On Oct 21, 2554 BE, at 11:57 AM, D_Tomas<tomasmeca at hotmail.com>  wrote:
>
>> My apologies for my vague comment.
>>
>> My data comprises 400.000 x 21 (17 explanatory variables, plus response
>> variable, plus two offsets).

This requires only 67MB to hold it in memory.
But maybe you have also other objects in your workspace?


>>
>> If I build the full model (only linear) I get:
>>
>> Error: cannot allocate vector of size 112.3 Mb


We cannot know how much memory is required. There are so many kinds of 
models. And even more important, it is extremely important to count each 
level of a factor as a separate variable -> think about the size of the 
design matrix, not only the size of the data!

>> I have a 4GB RAM laptop... Would i get any improvemnt on a 8GB computer

This may (only if both OS and R are 64-bit) or may not help.

Best,
Uwe Ligges





>> Many thanks,
>>
>>
>>
>>
>> --
>> View this message in context: http://r.789695.n4.nabble.com/glm-poisson-fitting-400-000-records-tp3925100p3925968.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.