[R-sig-ME] lme4 performance

[Gabor Grothendieck]

Regarding the slowness issue of lme4 vs. lme4.0 I have had some good
initial results getting lme4 to run as fast or faster than lme4.0 by
replacing the optimizer (which can be done without touching the code
of lme4 itself thanks to lme4's modular nature).  See my comment in
lme4 issue 150 on github

https://github.com/lme4/lme4/issues/150

where I have posted the results and the code need to try this.

If anyone has a mixed model dataset they would like to try out using
this approach please let me know.



On Sun, Jun 1, 2014 at 11:04 PM, Gabor Grothendieck
<ggrothendieck at gmail.com> wrote:
> Ben Bolker and I have been running some lmer timings on the lme4 package and
> the older lme4.0 package and I am posting some timings here for comment.
>
> This model was used:
>
>    lmer(y ~ service * dept + (1|s) + (1|d), InstEval)
>
> with lme4 from https://github.com/lme4/lme4/ and lme4.0 from
> http://download.r-forge.r-project.org/src/contrib/lme4.0_0.999999-4.tar.gz
>
> The newer lme4 is slower than the older lme4.0:
>
> - lme4 takes 51 sec for the above command vs. 33 sec for lme4.0
> - lme4 spends 37 sec of that in C/C++ optimizer whereas lme4.0 spends 26 sec
> - lme4 spends 33 sec of that performing cholesky factorization whereas
>   lme4.0 spends 14 sec
> - lme4 involves 57 iterations of bobyqa wheresd lme4.0 involves 14 iterations
>   of nlminb.  The iterations are the counts displayed if the actual argument
>   verbose = TRUE is added to the lmer call.
>
> We can reduce the 51 sec run time of lme4 to 45 sec (about 10% savings) if we
> add this actual argument to the lmer call above:
>    control = lmerControl(calc.derivs = FALSE)
>
> The fixed and random coefficients from the two packages were very close.
>
> system.time(lmer(...)) was used to get the lmer run time.
> UNIX time was used to get the time used when launched from littler and
> the percentages given by the google perftools profiler, also launched from
> littler, were applied to the UNIX time to get the other timings.
>
> R 3.0.3 running under Linux was used.
>
>
> --
> Statistics & Software Consulting
> GKX Group, GKX Associates Inc.
> tel: 1-877-GKX-GROUP
> email: ggrothendieck at gmail.com



-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com