[R-sig-ME] Reference ?

Phillip Alday me @end|ng |rom ph||||p@|d@y@com
Mon Dec 6 07:32:27 CET 2021

On 5/12/21 6:09 pm, Ben Bolker wrote:
>    More bad news:
>  * I can't point you to a specific, published/peer-reviewed 'best
> practices in mixed modeling' document; maybe someone else here can.

I don't really know of a single, short-ish document that covers this. A
lot of best practices follow from general statistical and numerical best
practices (see e.g. the "stable and can be trusted" bit that started
this -- you may have precise, well-defined estimates, but that's not
necessarily "stable" in the sense you want; perhaps confidence intervals
would be closer) and the underlying mathematics of mixed models.

There are a lot of introductory texts (both articles and books) on mixed
models in general, mixed models in particular problem domains (e.g.
ecology, cognitive science) and particular issues in mixed models
(random effects selection, etc.), but the quality of these varies
greatly. I recently came across one that has concrete reporting
recommendations (Meteyard and Davies 2020,
https://doi.org/10.1016/j.jml.2020.104092), but I think that they leave
out a lot of fine print (e.g., R2-like measures for mixed models are
problematic; the Kenward-Roger ddf approximation requires inverting a
large matrix, etc.).

>  * As Chris says, the lack of obvious numerical problems doesn't
> necessarily mean the model estimates are stable.
>  * The other issue with a small sample is that a lot of the inferential
> machinery of mixed models (p-values, confidence intervals, etc.) rests
> on asymptotic assumptions.
>  What kinds of "overfitting problems" did you have in mind? What would
> the symptoms be?
> On 12/5/21 6:20 PM, Chris Howden wrote:
>> Hi Sacha,
>> I'm not sure I really agree with that statement, although some may.
>> Say your sample size was really small, only 6. A simple model may
>> still fit, but I wouldn’t expect the parameters to be particularly
>> stable. Only 1 or 2 different datum could change everything.
>> I would also want to consider the SE of the parameter estimates. If
>> they are very large (compared to the parameter estimates), then this
>> is telling me the parameter estimates aren't very stable. Even though
>> the model converged.
>> Chris Howden B.Sc. (Hons)
>> Founding Partner
>> Data Analysis, Modelling and Training
>> Evidence Based Strategy/Policy Development, IP Commercialisation and
>> Innovation
>> (mobile) +61 (0) 410 689 945 | (skype) chris using trickysolutions.com.au
>> -----Original Message-----
>> From: R-sig-mixed-models <r-sig-mixed-models-bounces using r-project.org> On
>> Behalf Of Fernando Pedro Bruna Quintas
>> Sent: Sunday, 5 December 2021 6:55 AM
>> To: R-sig-mixed-models <r-sig-mixed-models using r-project.org>; varin sacha
>> <varinsacha using yahoo.fr>
>> Subject: Re: [R-sig-ME] Reference ?
>> Dear SV,
>> I can feel that you have a very promising research career just in
>> front of your eyes.
>> FB
>> ________________________________
>> De: R-sig-mixed-models <r-sig-mixed-models-bounces using r-project.org> en
>> nombre de varin sacha via R-sig-mixed-models
>> <r-sig-mixed-models using r-project.org>
>> Enviado: s bado, 4 de diciembre de 2021 20:47
>> Para: R-sig-mixed-models <r-sig-mixed-models using r-project.org>
>> Asunto: [R-sig-ME] Reference ?
>> Dear Mixed modelers experts,
>> I am looking for a reference to justify my sentence here below.
>> Many thanks for your help.
>> "The mixed model seemed well specified   it converged and had no
>> singular problem, no overfitting problem. So, even if the sample size
>> is quite small, the estimates are stable and can be trusted".
>> Best Regards,
>> SV
>> _______________________________________________
>> R-sig-mixed-models using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>     [[alternative HTML version deleted]]
>> _______________________________________________
>> R-sig-mixed-models using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

More information about the R-sig-mixed-models mailing list