[R-sig-ME] ML vs. REML to find a parsimonious mixed model

[Poe, John]

My take is that it usually doesn't matter and you can tell when it's likely
to before you've started running models based on sample size.

In practice, I almost always use ML unless I've got some specific reason to
try REML (either I have sample size issues or it is for pedagogical
purposes). REML will return less downwardly biased random effects so long
as your sample size is small so you might as well do it if it matters. I've
only seen them return materially different results in a couple of instances
when it wasn't a toy data set designed to produce differences. If you are
getting different p values for the deviance tests on random effects between
REML and ML then you should probably take that as a sign to be extra
paranoid. Most of my models are nonlinear so REML isn't strictly an option
in a lot of software anyway (it is possible but often not on offer). I say
do what you want as long as the deviance tests are still viable.




On Mon, Apr 23, 2018 at 5:38 PM, Maarten Jung <
Maarten.Jung at mailbox.tu-dresden.de> wrote:

>  Hi Christoph,
>
> No, I didn't.
> And I'm still very interested in what other mixed model experts/experienced
> mixed model users think about it.
> At the moment I tend to use REML for this purpose.
>
> Best,
> Maarten
>
> On Mon, Apr 23, 2018 at 4:20 PM, Christoph Huber <
> christoph.huber-huber at univie.ac.at> wrote:
>
> > Hi Maarten,
> >
> > Did you get any responses yet? I was facing the same problem and went for
> > REML eventually. But it still seems to me that this question does not
> (yet)
> > have a definite answer.
> >
> > Best,
> > Christoph
> >
> >
> >
> > Am 16.04.2018 um 12:00 schrieb r-sig-mixed-models-request at r-project.org:
> >
> > Send R-sig-mixed-models mailing list submissions to
> > r-sig-mixed-models at r-project.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> > or, via email, send a message with subject or body 'help' to
> > r-sig-mixed-models-request at r-project.org
> >
> > You can reach the person managing the list at
> > r-sig-mixed-models-owner at r-project.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of R-sig-mixed-models digest..."
> >
> >
> > Today's Topics:
> >
> >   1. ML vs. REML to find a parsimonious mixed model (Maarten Jung)
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Sun, 15 Apr 2018 13:00:08 +0200
> > From: Maarten Jung <Maarten.Jung at mailbox.tu-dresden.de>
> > To: Help Mixed Models <r-sig-mixed-models at r-project.org>
> > Subject: [R-sig-ME] ML vs. REML to find a parsimonious mixed model
> > Message-ID:
> > <CAHr4Dycsa1wmOXKKmDuGzrQi8pxgXq55iQxjEoEzFvyYNmvUvA at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > I want to use LRTs via anova() on fitted linear mixed models (merMod
> > objects) to find a parsimonious mixed model containing only variance
> > components supported by the data (e.g. Matuschek et al. 2017 [1], Bates
> et
> > al. 2015 [2]).
> > In this situation my focus is *only on the reduction of the random
> effects
> > part* of the models.
> > The aforementioned papers use ML instead of REML estimation within this
> > process. Douglas Bates seems to prefer ML model comparison due to the
> > skewed nature of the distribution of variance estimators [3] and the user
> > Wolfgang states that "the ML estimator usually has lower mean-squared
> error
> > (MSE) than the REML estimator" [4]. However, literally every textbook I
> > know suggests using REML estimation when comparing mixed models that
> differ
> > only in their random effect parts.
> >
> > What would you suggest in this particular situation? ML or REML?
> >
> > Best regards,
> > Maarten
> >
> > [1] https://arxiv.org/abs/1511.01864
> > [2] https://arxiv.org/abs/1506.04967
> > [3] https://stat.ethz.ch/pipermail/r-sig-mixed-models/2015q3/023750.html
> > [4] https://stats.stackexchange.com/a/48770
> >
> > [[alternative HTML version deleted]]
> >
> >
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > R-sig-mixed-models mailing list
> > R-sig-mixed-models at r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >
> >
> > ------------------------------
> >
> > End of R-sig-mixed-models Digest, Vol 136, Issue 26
> > ***************************************************
> >
> >
> > —
> > Dr. Christoph Huber-Huber
> > Center for Mind/Brain Sciences (CIMeC)
> > University of Trento
> > Corso Bettini 31
> > <https://maps.google.com/?q=Corso+Bettini+31+38068+
> Rovereto&entry=gmail&source=g>
> > 38068 Rovereto
> > <https://maps.google.com/?q=Corso+Bettini+31+38068+
> Rovereto&entry=gmail&source=g>
> > (TN), Italy
> >
> > e-mail: christoph.huberhuber at unitn.it
> >
> >
> >
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>



-- 




Thanks,
John


John Poe, Ph.D.
Postdoctoral Scholar / Research Methodologist
Center for Public Health Services & Systems Research
University of Kentucky
www.johndavidpoe.com

	[[alternative HTML version deleted]]