[R-sig-ME] Missing values in lmer vs. HLM

landon hurley ljrhurley at gmail.com
Sun Jul 5 06:35:24 CEST 2015

Hash: SHA512

On 07/05/2015 12:14 AM, Phillip Alday wrote:
> On Sat, 2015-07-04 at 21:21 +0200, Karl Ove Hufthammer wrote:
>> Den 04. juli 2015 18:18, Douglas Bates skreiv:
>>> Having said all this I will admit that the original sin, the 
>>> "REML" criterion, was committed by statisticians.  In retrospect 
>>> I wish that we had not incorporated that criterion into the nlme 
>>> and lme4 packages but, at the time we wrote them, our work would 
>>> have been dismissed as wrong if our answers did not agree with 
>>> SAS PROC MIXED, etc.  So we opted for bug-for-bug compatibility 
>>> with existing software.
>> Hm. I find this statement surprising. I was under the impression 
>> REML is *preferred* to ML in many situations (e.g. in simple
>> random intercept models with few observations for each random
>> intercept), and that *ML estimation* may result in severe bias. Do
>> you consider maximising the REML criterion as a bug?
> This was my question as well. My understanding was that REML, like 
> Bessel's correction for the sample variance, was motivated by bias in
> the maximum-likelihood estimator for small numbers of observations.
> The corrected estimator is in both cases no longer the MLE, so that
> the ML part is bit of a misnomer, but if you take "residualized"
> expansion of RE instead of "restricted", then REML seems more like a
> function of ML and not a "type" of ML.
> IIRC, the default in MixedModels.jl is now ML -- have you changed 
> your opinion about the utility of REML? Is there some type of weird 
> paradoxical situation with REML like with Bessel's correction -- the
>  variance estimates are no longer biased, but the s.d. estimates
> are?
> Or is the original sin the use of the name REML when REML is no 
> longer *the* maximum likelihood?

I had assumed that he would have responded by now, but it is a holiday
in the US. The position Bates is taking is explained (I think) in his
2010 report
lme4: Mixed effects modelling with R in Section 5.5 `The REML
Criterion', roughly page 123-124 in the pdf [0]. It's a short read, but
the most relevant bit I think is:

> The argument for preferring σ_R to σ_L as an estimate of σ**2 is
> that the numerator in both estimates is the sum of squared
> residuals at β and, although the residual vector, yobs − Xβ , is an
> n-dimensional vector, the residual at θ satisfies p linearly
> independent constraints, X**{T} (yobs − Xβ ) = 0. That is, the residual
> at θ is the projection of the observed response vector, yobs , into
> an (n − p)-dimensional linear subspace of the n-dimensional response
> space. The estimate σR takes into account the fact that σ**2 is
> estimated from residuals that have only n − p degrees of freedom.
> Another argument often put forward for REML estimation is that σ_R is 
> an unbiased estimate of σ**2 , in the sense that the expected value of
> the estimator is equal to the value of the parameter. However, 
> determining the expected value of an estimator involves integrating 
> with respect to the density of the estimator and we have seen that 
> densities of estimators of variances will be skewed, often highly 
> skewed. It is not clear why we should be interested in the expected 
> value of a highly skewed estimator. If we were to transform to a
> more symmetric scale, such as the estimator of the standard deviation
> or the estimator of the logarithm of the standard deviation, the
> REML estimator would no longer be unbiased. Furthermore, this
> property of unbiasedness of variance estimators does not generalize
> from the linear regression model to linear mixed models. This is all
> to say that the distinction between REML and ML estimates of
> variances and variance components is probably less important that
> many people believe.



> Best, Phillip Alday _______________________________________________ 
> R-sig-mixed-models at r-project.org mailing list 
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

- -- 
Violence is the last refuge of the incompetent.
Version: GnuPG v2.0.17 (GNU/Linux)


More information about the R-sig-mixed-models mailing list