[R-sig-ME] [Fwd: Re: Fwd: same old question - lme4 and p-values]
Ben Bolker
bolker at zoo.ufl.edu
Sun Apr 13 03:03:37 CEST 2008
(resending -- first try bounced)
Also note that in the long thread on the R wiki
(wiki.r-project.org, search for "bates mixed" or some such --
I can't get through to it right now) DB suggests an
test for a composite hypothesis a_1=a_2=...=a_n=0
along with R code to do it ...
Andrew Robinson wrote:
> On Sat, Apr 12, 2008 at 02:02:09PM +0200, Reinhold Kliegl wrote:
>> On Fri, Apr 11, 2008 at 3:10 PM, Kevin E. Thorpe
>> <kevin.thorpe at utoronto.ca> wrote:
>>> This has been a very interesting thread. However, I'm still
>>> wrestling with what to do for a fixed-effect that has more than
>>> one degree of freedom.
>>>
>>> In the data I'm analyzing, I have three groups to compare.
>>>
>>> So, I can get CIs for the two parameters, but that is a bit
>>> problematic for assessing an overall difference.
>>>
>>> Is it valid to do the following? Estimate the parameters using both
>>> ML and REML. If the estimates show good agreement, is that sufficient
>>> evidence to conclude the ML procedure is converging and that I can
>>> use a likelihood ratio test for the fixed effect?
>>>
>> I assume you refer to using anova(fm1, fm2) with fm1 fitting the model
>> without the fixed effect. This a comparison of nested models, so a
>> likelihood ratio test can be defined for ML fits only. Note, however,
>> that Pinheiro & Bates (2000, p. 87, 2.4.2) "do not recommend using
>> such tests"; "not" is set in bold face. They show that such tests tend
>> to be anti-conservative, especially if the number of parameters
>> removed is large relative to the number of observations. Assuming you
>> have a decent number of total observations, you may be fine.
>> Alternatively, you may want to run a simulation for your situation;
>> you will also find R-code examples in the P&B section.
>
> I agree with Reinhold's position, here. I also note in passing that
> Doug uses this strategy to test the fixed effects in the cake data
> (see ?cake). Doug, does the cake data analysis represent a softening
> on your position or a place-filler?
>
>> My first reaction to your email was: Why is he only interested in the
>> overall effect of a fixed factor and not in specific comparisons
>> between its levels? After Andrew's comment to an earlier post, I
>> understand that there are such situations where you just want to
>> control for an aspect of the design that is not in the focus of your
>> theoretical concerns (e.g., in ecology you may have three sites that
>> could be characterized as levels of a fixed factor or as a sample from
>> a random factor). Perhaps your fixed factor may also be better
>> conceptualized as a random factor. In a way, you just want to control
>> for the variance contributed by this factor. If this applies to your
>> data, then you may be better off to specify your fixed factor as a
>> random factor. Then, your anova(fm1, fm2) compares nested models that
>> differ only in the random-effects part. In this case the likelihood
>> ratio test can be used with models fit by REML. These tests tend to be
>> conservative (Pinheiro & Bates, 2000, p. 2.4.1; following up on Stram
>> & Lee, 1994). So if your ANOVA statistic is significant, you are on
>> the save side; if not, you do not know. Also keep in mind, that random
>> effects with few units may generate problems for model convergence.
>
> That's an interesting idea, even if the interpretation is intended to
> be a fixed factor. It might work to a certain order of approximation,
> but I'm not clear how the math would play out. Some simulations might
> provide a measure of comfort in individual situations.
>
> Best wishes,
>
> Andrew
>
More information about the R-sig-mixed-models
mailing list