[R-sig-ME] [Fwd: Re: Fwd: same old question - lme4 and p-values]

[Ben Bolker]

   (resending -- first try bounced)

    Also note that in the long thread on the R wiki
(wiki.r-project.org, search for "bates mixed" or some such --
I can't get through to it right now) DB suggests an
test for a composite hypothesis a_1=a_2=...=a_n=0
along with R code to do it ...

Andrew Robinson wrote:
> On Sat, Apr 12, 2008 at 02:02:09PM +0200, Reinhold Kliegl wrote:
>> On Fri, Apr 11, 2008 at 3:10 PM, Kevin E. Thorpe
>> <kevin.thorpe at utoronto.ca> wrote:
>>> This has been a very interesting thread.  However, I'm still
>>>  wrestling with what to do for a fixed-effect that has more than
>>>  one degree of freedom.
>>>
>>>  In the data I'm analyzing, I have three groups to compare.
>>>
>>>  So, I can get CIs for the two parameters, but that is a bit
>>>  problematic for assessing an overall difference.
>>>
>>>  Is it valid to do the following?  Estimate the parameters using both
>>>  ML and REML.  If the estimates show good agreement, is that sufficient
>>>  evidence to conclude the ML procedure is converging and that I can
>>>  use a likelihood ratio test for the fixed effect?
>>>
>> I assume you refer to using anova(fm1, fm2) with fm1 fitting the model
>> without the fixed effect. This a comparison of nested models, so a
>> likelihood ratio test can be defined for ML fits only. Note, however,
>> that Pinheiro & Bates (2000, p. 87, 2.4.2) "do not recommend using
>> such tests"; "not" is set in bold face. They show that such tests tend
>> to be anti-conservative, especially if the number of parameters
>> removed is large relative to the number of observations. Assuming you
>> have a decent number of total observations, you may be fine.
>> Alternatively, you may want to run a simulation for your situation;
>> you will also find R-code examples in the P&B section.
> 
> I agree with Reinhold's position, here.  I also note in passing that
> Doug uses this strategy to test the fixed effects in the cake data
> (see ?cake).  Doug, does the cake data analysis represent a softening
> on your position or a place-filler?
>  
>> My first reaction to your email was: Why is he only interested in the
>> overall effect of a fixed factor and not in specific comparisons
>> between its levels? After Andrew's comment to an earlier post, I
>> understand that there are such situations where you just want to
>> control for an aspect of the design that is not in the focus of your
>> theoretical concerns (e.g., in ecology you may have three sites that
>> could be characterized as levels of a fixed factor or as a sample from
>> a random factor). Perhaps  your fixed factor may also be better
>> conceptualized as a random factor. In a way, you just want to control
>> for the variance contributed by this factor. If this applies to your
>> data, then you may be better off to specify your fixed factor as a
>> random factor. Then, your anova(fm1, fm2) compares nested models that
>> differ only in the random-effects part. In this case the likelihood
>> ratio test can be used with models fit by REML. These tests tend to be
>> conservative (Pinheiro & Bates, 2000, p. 2.4.1; following up on Stram
>> & Lee, 1994). So if your ANOVA statistic is significant, you are on
>> the save side; if not, you do not know. Also keep in mind, that random
>> effects with few units may generate problems for model convergence.
> 
> That's an interesting idea, even if the interpretation is intended to
> be a fixed factor.  It might work to a certain order of approximation,
> but I'm not clear how the math would play out.  Some simulations might
> provide a measure of comfort in individual situations.
> 
> Best wishes,
> 
> Andrew
>