[R-sig-ME] Missing values in lmer vs. HLM

[landon hurley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 07/05/2015 12:14 AM, Phillip Alday wrote:
> On Sat, 2015-07-04 at 21:21 +0200, Karl Ove Hufthammer wrote:
>> Den 04. juli 2015 18:18, Douglas Bates skreiv:
>>> Having said all this I will admit that the original sin, the 
>>> "REML" criterion, was committed by statisticians.  In retrospect 
>>> I wish that we had not incorporated that criterion into the nlme 
>>> and lme4 packages but, at the time we wrote them, our work would 
>>> have been dismissed as wrong if our answers did not agree with 
>>> SAS PROC MIXED, etc.  So we opted for bug-for-bug compatibility 
>>> with existing software.
>> 
>> Hm. I find this statement surprising. I was under the impression 
>> REML is *preferred* to ML in many situations (e.g. in simple
>> random intercept models with few observations for each random
>> intercept), and that *ML estimation* may result in severe bias. Do
>> you consider maximising the REML criterion as a bug?
>> 
> 
> This was my question as well. My understanding was that REML, like 
> Bessel's correction for the sample variance, was motivated by bias in
> the maximum-likelihood estimator for small numbers of observations.
> The corrected estimator is in both cases no longer the MLE, so that
> the ML part is bit of a misnomer, but if you take "residualized"
> expansion of RE instead of "restricted", then REML seems more like a
> function of ML and not a "type" of ML.
> 
> IIRC, the default in MixedModels.jl is now ML -- have you changed 
> your opinion about the utility of REML? Is there some type of weird 
> paradoxical situation with REML like with Bessel's correction -- the
>  variance estimates are no longer biased, but the s.d. estimates
> are?
> 
> 
> Or is the original sin the use of the name REML when REML is no 
> longer *the* maximum likelihood?
> 

I had assumed that he would have responded by now, but it is a holiday
in the US. The position Bates is taking is explained (I think) in his
2010 report
lme4: Mixed effects modelling with R in Section 5.5 `The REML
Criterion', roughly page 123-124 in the pdf [0]. It's a short read, but
the most relevant bit I think is:

> The argument for preferring σ_R to σ_L as an estimate of σ**2 is
> that the numerator in both estimates is the sum of squared
> residuals at β and, although the residual vector, yobs − Xβ , is an
> n-dimensional vector, the residual at θ satisfies p linearly
> independent constraints, X**{T} (yobs − Xβ ) = 0. That is, the residual
> at θ is the projection of the observed response vector, yobs , into
> an (n − p)-dimensional linear subspace of the n-dimensional response
> space. The estimate σR takes into account the fact that σ**2 is
> estimated from residuals that have only n − p degrees of freedom.
> 
> Another argument often put forward for REML estimation is that σ_R is 
> an unbiased estimate of σ**2 , in the sense that the expected value of
> the estimator is equal to the value of the parameter. However, 
> determining the expected value of an estimator involves integrating 
> with respect to the density of the estimator and we have seen that 
> densities of estimators of variances will be skewed, often highly 
> skewed. It is not clear why we should be interested in the expected 
> value of a highly skewed estimator. If we were to transform to a
> more symmetric scale, such as the estimator of the standard deviation
> or the estimator of the logarithm of the standard deviation, the
> REML estimator would no longer be unbiased. Furthermore, this
> property of unbiasedness of variance estimators does not generalize
> from the linear regression model to linear mixed models. This is all
> to say that the distinction between REML and ML estimates of
> variances and variance components is probably less important that
> many people believe.


best,

landon
[0]:
www.researchgate.net/publictopics.PublicPostFileLoader.html?id=53326f19d5a3f206348b45af&key=6a85e53326f199010f

> Best, Phillip Alday _______________________________________________ 
> R-sig-mixed-models at r-project.org mailing list 
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> 


- -- 
Violence is the last refuge of the incompetent.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)

iQIcBAEBCgAGBQJVmLQMAAoJEDeph/0fVJWs21AP/RPMNrQDdkd67eQWfb9jhbTZ
VIfZDNAt088n8XednUzk6BQlVUirb8akqdVq8YqtdaCotdP5dxXjRr30hO72FeRj
rvLWcZXtDVuNwOFZA44Aw2YplwD1sU+G+vMLOsPD4BrBRfByY5FkkX2lTliQjbVK
eYNi57977w/AE7y48OTprwBdkNkWjTQTrnKAsglBtOlnC+x8TdD8J0WfaZCfI1CP
iUN6298pdSNWm+vAWaFCeq6Wig8o2kYGONd1RBDzGidbcy5CiQebuJYZ8cU5zl4V
X34alL6cX+9qDLWbi4jYz1/3lG1U4NsCKhs6fM7imxOFV9XXtsZTr16xmHbkc6B+
daNAbln/SHKoCpuvGiO+IL/H8y8W1DLyJk7sRlb7tOuDHViBCOPLwPZj71aJFFab
qY40JvxdV0rputOeg5OtTyqROxCwtgqKWDaGcli1Dpcrca2aE7qNpPdMjxmnJN+j
RDaCkHflJuwHifkupg7qSfLa2zbBf4KirfGnOsoeicZPF4s7BmDLU28fMlhAqEly
Dbg3niPaW4/X3X69yrkw5XG46uftRGlSUdl/eMWWBUVy3o5oAkAgxpQl/49Md0CX
SyYyv+Iaa9NPtgI828NYfw59xsW98hTnNNtG+D0V4nN/1d29M/KM6LspxV9Nu64b
XNno0S7U16mSu1KhvIwY
=7uyi
-----END PGP SIGNATURE-----