[R-sig-ME] lmer() fit

[Ben Pelzer]

Hi,

I used Julia to do a similar job some months ago with about 35 fixed 
effects and one random slope. On my PC  (older type i7 processor, 
windows 7, 12 Gb memory) Julia was about 5 to 6 times as fast as lmer in 
R. Since I had to estimate many models, one after the other, I could 
also use the parallel processing option in Julia, which resulted in 
about 20 times as fast per model as in R. To be honest: I did not use 
the same facility in R, which exists, but just to give you an idea.

Regards, Ben.


On 17-5-2016 21:43, Ben Bolker wrote:
>    This doesn't seem like a big deal.  The following fit takes about 4.5
> seconds on my Macbook Pro.
>
> library(lme4)
> set.seed(101)
> nRE <- 20000
> nobs <- 250000
> dd <- data.frame(f=sample(1:nRE,size=nobs,replace=TRUE),
>                   x=rnorm(nobs))
> dd$y <- simulate(~x+(1|f),
>           newparams=list(beta=c(1,2),theta=1,sigma=1),
>           newdata=dd,
>           family=gaussian,
>           seed=102)[[1]]
> system.time(fit <- lmer(y~x+(1|f),
>                data=dd))
> fixef(fit)
> VarCorr(fit)
>
>
>     If you have lots of fixed effects or very complex random effects,
> things could get a bit slower.  If you have a *much* bigger problem than
> this -- or if you're going to want to this sort of thing thousands of
> times in a row and 4.5 seconds is too slow -- you might want to talk to
> Doug Bates about the MixedModels package for Julia ...
>
>
> On 16-05-17 02:23 PM, Chaitanya Acharya wrote:
>> Hi, Apologies for a very non-specific question. Any idea how many
>> random effects could lmer() reasonably fit? I am thinking of a
>> situation where I want to fit ~20k random effects with ~250k
>> observations. What kind of issues should I foresee?
>>
>> Thanks, Chuck _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models