[R-sig-ME] Why do lme() models runs so fast and converge as compared to lmer()?

[Santosh Srinivas]

Thank you, Dr. Bolker. The advice to set calc.derivs = FALSE is much appreciated.

We tried glmmTMB() for the same model and found it quite fast. Models converged with default optimizers.

Following this, we also tried Julia and were surprised by the orders-of-magnitude improvement in speed.

We are tending to stick to R and use glmmTMB(). We hope that academic journals are generally receptive to using this package for hypothesis testing. From a cursory Google scholar search, it appears that glmmTMB() is used/cited less often (and Julia's MixedModels far and few) as compared to lme() and lmer().

Assuming that models are specified identically, I am hoping the results for hypothesis testing (e.g., significance of a coefficient of theoretical importance) should be identical regardless of whether one uses R's lme(), lmer(), or glmmTMB(), or Julia's MixedModels().

On this note, anything that you suggest we state explicitly in the journal manuscript when presenting the results using glmmTMB() as opposed to lme() or lmer()?

Thanks!
sbs
________________________________
From: R-sig-mixed-models <r-sig-mixed-models-bounces using r-project.org> on behalf of Ben Bolker <bbolker using gmail.com>
Sent: Monday, September 18, 2023 2:11 AM
To: r-sig-mixed-models using r-project.org <r-sig-mixed-models using r-project.org>
Subject: Re: [R-sig-ME] Why do lme() models runs so fast and converge as compared to lmer()?

    This is mildly surprising but not impossible. I haven't looked into
it/run any experiments yet, but:

  * lme handles a more restricted range of model types, so it's possible
that its algorithm is faster on simple examples;

  * 'doesn't converge' may not be the issue you think it is.  I would
improve performance and turn off convergence checking that is known to
be dodgy for large data sets via control = lmerControl(calc.derivs =
FALSE)), and more generally try these suggestions:
https://rdrr.io/cran/lme4/f/vignettes/lmerperf.Rmd

   cheers
    Ben Bolker

On 2023-09-17 5:21 a.m., Santosh Srinivas wrote:
> Hi, I am running the following similar models using lme() and lmer():
>
> f = y ~ x + year * post_event + (year|user_id)
>
> where,
>
>    *   year (integer) ranges from 0 (for the year 2010) to 10 (for the year2020);
>    *   post_event (factor variable) is 1 for years 2013 onwards, and 0 otherwise;
>    *   Number of Observations: 3586633; and
>    *   Number of Groups: 1109.
>
> The lme4's lmer() runs for several minutes and never succeeds to converge with any optimizer I try, whereas the former seems to converge in a few minutes.
>
> Not sure if I am doing anything wrong, or whether such convergence and performances are generally expected of lme().
>
> Request your help.
>
> Thanks & regards,
> sbs
>
>
>
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

--
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
(Acting) Graduate chair, Mathematics & Statistics
 > E-mail is sent at my convenience; I don't expect replies outside of
working hours.

_______________________________________________
R-sig-mixed-models using r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

	[[alternative HTML version deleted]]