[R-sig-ME] Why do lme() models runs so fast and converge as compared to lmer()?

[Ben Bolker]

    This is mildly surprising but not impossible. I haven't looked into 
it/run any experiments yet, but:

  * lme handles a more restricted range of model types, so it's possible 
that its algorithm is faster on simple examples;

  * 'doesn't converge' may not be the issue you think it is.  I would 
improve performance and turn off convergence checking that is known to 
be dodgy for large data sets via control = lmerControl(calc.derivs = 
FALSE)), and more generally try these suggestions: 
https://rdrr.io/cran/lme4/f/vignettes/lmerperf.Rmd

   cheers
    Ben Bolker

On 2023-09-17 5:21 a.m., Santosh Srinivas wrote:
> Hi, I am running the following similar models using lme() and lmer():
> 
> f = y ~ x + year * post_event + (year|user_id)
> 
> where,
> 
>    *   year (integer) ranges from 0 (for the year 2010) to 10 (for the year2020);
>    *   post_event (factor variable) is 1 for years 2013 onwards, and 0 otherwise;
>    *   Number of Observations: 3586633; and
>    *   Number of Groups: 1109.
> 
> The lme4's lmer() runs for several minutes and never succeeds to converge with any optimizer I try, whereas the former seems to converge in a few minutes.
> 
> Not sure if I am doing anything wrong, or whether such convergence and performances are generally expected of lme().
> 
> Request your help.
> 
> Thanks & regards,
> sbs
> 
> 
> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-- 
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
(Acting) Graduate chair, Mathematics & Statistics
 > E-mail is sent at my convenience; I don't expect replies outside of 
working hours.