[R-sig-eco] nlme model specification

[Kingsford Jones]

On Mon, May 26, 2008 at 8:22 AM, Nicholas Lewin-Koh <nikko at hailmail.net> wrote:
> Hi,
> Last month, or so, Doug was talking about comparing models
> using likelihood ratio tests, anova(m1,m2) and pointed out
> that the way things are calculated in lmer the ML and REML estimates
> are equivalent. I am not sure if this is because the bias in the REML
> estimates cancels out or if the estimation procedures in lmer get them
> to the same place. I wish I could find the post. However this has to do
> with
> the likelihood vs. restricted profile likelihood for the whole model
> and may not be relevant to the parameter estimates.
>

I certainly wouldn't want to imply that LRTs are not useful for
mixed-model comparisons, but I think there are some issues that users
should at least be aware of (if lmer has overcome any of these I'd be
interested in hearing about it):

- LRTs aren't valid to compare REML fits with different fixed effects
because REML essentially maximizes A'Y where E[A'Y] = 0, so changing
the fixed effects changes A' which changes the data making the
likelihoods non-comparable.
- Pinheiro and Bates (2000, pg 87-88) recommend LRTs with the standard
X^2 distribution not be used to compare ML fits with different fixed
effects because the tests can be very "anticonservative" --
particularly as the number of parameters being removed becomes large
relative to the number of observations.
- LRTs for differences in the random part of the model when the fixed
effects are the same can be conservative due to the null value of 0
being on the edge of the variance parameter space.
- It seems the issue of counting the number of parameters being
estimated will be an issue when comparing models that differ in their
random effects (i.e. for determining the df of the X^2).

Kingsford Jones





>> ------------------------------
>>
>> Message: 2
>> Date: Sun, 25 May 2008 11:31:39 -0700
>> From: "Kingsford Jones" <kingsfordjones at gmail.com>
>> Subject: Re: [R-sig-eco] nlme model specification
>> To: "Ruben Roa Ureta" <rroa at udec.cl>
>> Cc: r-sig-ecology at r-project.org
>> Message-ID:
>>       <2ad0cc110805251131q48629679oc8b29eb2296fe320 at mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> On Sun, May 25, 2008 at 7:39 AM, Ruben Roa Ureta <rroa at udec.cl> wrote:
>> > [snip]
>> >
>> >> I think you're right that there is some shaky ground here, and Doug
>> >> Bates has pointed out some issues on the R-sig-mixed-models list (I
>> >> can't seem to find the thread right now).  One of the issues is that
>> >> mixed models are generally fit with REML, which is not ML and
>> >> therefore does not technically conform to the derivations of the *IC.
>> >> If you fit a mixed model with ML instead, bias is introduced.
>> >
>> > Bates think that the maximum log likelihood is not a problem with mixed
>> > models when fit using ML:
>> > http://finzi.psych.upenn.edu/R/Rhelp02a/archive/117488.html
>> > though he does see a problem with the counting of parameters.
>> > Sorry if I am a bit lost coming late to this thread.
>>
>>
>> Thanks Rubin -- that's the link I was looking for.
>>
>> When I wrote the paragraph above I was hoping I wasn't misrepresenting
>> what Bates said, and I don't think I did.  The problem is that if you
>> fit with ML you introduce downward bias in the estimates of the
>> variance components -- that's why REML is the default method.  And, as
>> you mentioned, you still have the problem of specifying the number of
>> parameters estimated.  How much all of this matters, I'm not sure --
>> perhaps the paper Jarrett referred to will help clarify things.
>>
>> best,
>> Kingsford
>>
>> >
>> > _______________________________________________
>> > R-sig-ecology mailing list
>> > R-sig-ecology at r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
>> >
>>
>>
>>
>