[R-sig-ME] Missing values in lmer vs. HLM
landon hurley
ljrhurley at gmail.com
Sun Jul 5 06:35:24 CEST 2015
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
On 07/05/2015 12:14 AM, Phillip Alday wrote:
> On Sat, 2015-07-04 at 21:21 +0200, Karl Ove Hufthammer wrote:
>> Den 04. juli 2015 18:18, Douglas Bates skreiv:
>>> Having said all this I will admit that the original sin, the
>>> "REML" criterion, was committed by statisticians. In retrospect
>>> I wish that we had not incorporated that criterion into the nlme
>>> and lme4 packages but, at the time we wrote them, our work would
>>> have been dismissed as wrong if our answers did not agree with
>>> SAS PROC MIXED, etc. So we opted for bug-for-bug compatibility
>>> with existing software.
>>
>> Hm. I find this statement surprising. I was under the impression
>> REML is *preferred* to ML in many situations (e.g. in simple
>> random intercept models with few observations for each random
>> intercept), and that *ML estimation* may result in severe bias. Do
>> you consider maximising the REML criterion as a bug?
>>
>
> This was my question as well. My understanding was that REML, like
> Bessel's correction for the sample variance, was motivated by bias in
> the maximum-likelihood estimator for small numbers of observations.
> The corrected estimator is in both cases no longer the MLE, so that
> the ML part is bit of a misnomer, but if you take "residualized"
> expansion of RE instead of "restricted", then REML seems more like a
> function of ML and not a "type" of ML.
>
> IIRC, the default in MixedModels.jl is now ML -- have you changed
> your opinion about the utility of REML? Is there some type of weird
> paradoxical situation with REML like with Bessel's correction -- the
> variance estimates are no longer biased, but the s.d. estimates
> are?
>
>
> Or is the original sin the use of the name REML when REML is no
> longer *the* maximum likelihood?
>
I had assumed that he would have responded by now, but it is a holiday
in the US. The position Bates is taking is explained (I think) in his
2010 report
lme4: Mixed effects modelling with R in Section 5.5 `The REML
Criterion', roughly page 123-124 in the pdf [0]. It's a short read, but
the most relevant bit I think is:
> The argument for preferring σ_R to σ_L as an estimate of σ**2 is
> that the numerator in both estimates is the sum of squared
> residuals at β and, although the residual vector, yobs − Xβ , is an
> n-dimensional vector, the residual at θ satisfies p linearly
> independent constraints, X**{T} (yobs − Xβ ) = 0. That is, the residual
> at θ is the projection of the observed response vector, yobs , into
> an (n − p)-dimensional linear subspace of the n-dimensional response
> space. The estimate σR takes into account the fact that σ**2 is
> estimated from residuals that have only n − p degrees of freedom.
>
> Another argument often put forward for REML estimation is that σ_R is
> an unbiased estimate of σ**2 , in the sense that the expected value of
> the estimator is equal to the value of the parameter. However,
> determining the expected value of an estimator involves integrating
> with respect to the density of the estimator and we have seen that
> densities of estimators of variances will be skewed, often highly
> skewed. It is not clear why we should be interested in the expected
> value of a highly skewed estimator. If we were to transform to a
> more symmetric scale, such as the estimator of the standard deviation
> or the estimator of the logarithm of the standard deviation, the
> REML estimator would no longer be unbiased. Furthermore, this
> property of unbiasedness of variance estimators does not generalize
> from the linear regression model to linear mixed models. This is all
> to say that the distinction between REML and ML estimates of
> variances and variance components is probably less important that
> many people believe.
best,
landon
[0]:
www.researchgate.net/publictopics.PublicPostFileLoader.html?id=53326f19d5a3f206348b45af&key=6a85e53326f199010f
> Best, Phillip Alday _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
- --
Violence is the last refuge of the incompetent.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
iQIcBAEBCgAGBQJVmLQMAAoJEDeph/0fVJWs21AP/RPMNrQDdkd67eQWfb9jhbTZ
VIfZDNAt088n8XednUzk6BQlVUirb8akqdVq8YqtdaCotdP5dxXjRr30hO72FeRj
rvLWcZXtDVuNwOFZA44Aw2YplwD1sU+G+vMLOsPD4BrBRfByY5FkkX2lTliQjbVK
eYNi57977w/AE7y48OTprwBdkNkWjTQTrnKAsglBtOlnC+x8TdD8J0WfaZCfI1CP
iUN6298pdSNWm+vAWaFCeq6Wig8o2kYGONd1RBDzGidbcy5CiQebuJYZ8cU5zl4V
X34alL6cX+9qDLWbi4jYz1/3lG1U4NsCKhs6fM7imxOFV9XXtsZTr16xmHbkc6B+
daNAbln/SHKoCpuvGiO+IL/H8y8W1DLyJk7sRlb7tOuDHViBCOPLwPZj71aJFFab
qY40JvxdV0rputOeg5OtTyqROxCwtgqKWDaGcli1Dpcrca2aE7qNpPdMjxmnJN+j
RDaCkHflJuwHifkupg7qSfLa2zbBf4KirfGnOsoeicZPF4s7BmDLU28fMlhAqEly
Dbg3niPaW4/X3X69yrkw5XG46uftRGlSUdl/eMWWBUVy3o5oAkAgxpQl/49Md0CX
SyYyv+Iaa9NPtgI828NYfw59xsW98hTnNNtG+D0V4nN/1d29M/KM6LspxV9Nu64b
XNno0S7U16mSu1KhvIwY
=7uyi
-----END PGP SIGNATURE-----
More information about the R-sig-mixed-models
mailing list