[R] How to adjust the stack size of R

[Uwe Ligges]

On 02.04.2012 10:04, Christofer Bogaso wrote:
> Hi Uwe, you said 'You need at least 76252 obs and that means the
> design matrix needs>  46
> Gbyte! '
>
> Can you please explain me how you come up with these 2 numbers? Is the
> number '76252  = 76251 + 1 (1 for the intercept in the model)' somehow
> related with the Saturated model?

If you want to estimate 76252 coefficients, you will need *at least* as 
many observations.
hence I took at as a lower limit and calculated 76252*76252*8 for the 
required amount of storage given we expect floats in double precision as 
input. Is these are factors with more than 2 levels, there are even more 
parameters o estimate.

Uwe Ligges


>
> Thanks,
>
> On Sun, Apr 1, 2012 at 10:46 PM, Uwe Ligges
> <ligges at statistik.tu-dortmund.de>  wrote:
>>
>>
>> On 01.04.2012 08:46, Xiaobo Gu wrote:
>>>
>>> 2012/3/31 Uwe Ligges<ligges at statistik.tu-dortmund.de>
>>>
>>>>
>>>>
>>>> On 30.03.2012 03:16, Xiaobo Gu wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I got a stack overflow error when training a glm model with a very long
>>>>> formula.
>>>>>
>>>>
>>>> I just tried with a formula of length 1000. How long was yours?
>>>> Which version of R? Where is the repdroducible example?
>>>>
>>>>
>>> We have a glm formula with 76251 terms, the text of the formula is about
>>> 150K, we are using R 2.14.2
>>>
>>> Xiaobo Gu
>>>
>>
>>
>> You would need do compile your own copy of R and increase the stack size, on
>> the slow machines around here, it takes even hours just to parse the
>> formula.
>>
>> Aynway, I canmnot believe you have a sufficient number of observations in
>> the end:
>>
>> You need at least 76252 obs and that means the design matrix needs>  46
>> Gbyte! Hence a sensible calculation is not really possible unless you have
>> really big machines around.
>>
>> Best,
>> Uwe Ligges
>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.