[R-sig-ME] test significance of single random effect

[Achaz von Hardenberg]

Dear all,
I am coming back on the recent issue on how to test the significance  
of a single random term in linear mixed models...

In Zuur et al.  "Mixed Models and Extentions in Ecology with R"  
Springer, 2009, the authors suggest to compare a lme model (with the  
random effect) with a gls model with the same fixed effects structure,  
and then compare the AICs of the two models or using a likelihood  
ratio test via the ANOVA comand (pages 122 - 128).

I would be interested in hearing the opinion of other members of the  
list on this approach...

Thanks a lot,

Achaz


On 17 Nov 2009, at 20:41, Tom Van Dooren wrote:

> With REML=FALSE RLRsim seems to work fine in R 2.10, if I use the  
> design matrix and Zt as arguments in LRTSim().
> Otherwise I didn't get useful results out.
>
> That's not too much of a problem.
> It is not difficult to simulate the null model without random  
> effect, extract logLikelihoods from the (generalized) mixed model  
> and the (generalized) linear model fitted to those pseudo-data, to  
> calculate a distribution of likelihood ratios,
> which are then maybe off by a constant.
> What I was mainly uncertain about, is whether the log-likelihood of  
> a mixed model (also fitted to data simulated from the null model  
> without random effect),
> can be used as a statistic itself?
> The answer might be a simple NO! of course, or something more  
> involved...
>
> Tom
>
>
> Douglas Bates wrote:
>> On Tue, Nov 17, 2009 at 3:49 AM, Matthias Gralle
>> <matthias_gralle at eva.mpg.de> wrote:
>>
>>> I had basically the same problem a short time ago, and resorted to  
>>> lme
>>> instead of lmer, because one can directly compare lme and lm  
>>> objects using
>>> anova(). Is that OK, or is this feature of lme depreciated ?
>>>
>>
>> Is that not possible for linear mixed-effects models fit by lmer  
>> using
>> REML = FALSE? (Occasionally I lose track of what can be done in
>> different versions of lme4.)  You don't want to compare an lmer model
>> fit by REML with the log-likelihood of an lm model but you should be
>> able to compare likelihoods (subject to the caveat that the p-value
>> for the likelihood ratio test on the boundary of the parameter space
>> is conservative).
>>
>>
>>> Ben Bolker wrote:
>>>
>>>> Have you tried the RLRsim package??
>>>>
>>>> Tom Van Dooren wrote:
>>>>
>>>>
>>>>> I tried to find an easy way to test whether the random effect  
>>>>> would be
>>>>> significant in a (generalized) mixed model with a single random  
>>>>> effect.
>>>>> It annoyed me that log-likelihoods of lm or glm and lmer are not
>>>>> necesarily directly comparable -> trouble with calculating  
>>>>> likelihood
>>>>> ratios.
>>>>> What do members of this list think of the following simulation  
>>>>> approach?
>>>>> It basically amounts to simulating a distribution for the log  
>>>>> likelihood,
>>>>> given the null hypothesis that there is no random effect  
>>>>> variance and that
>>>>> the fixed effect model is correct.
>>>>>
>>>>>
>>>>> library(lme4)
>>>>> mm1 <- lmer(Reaction ~ Days + (1|Subject), sleepstudy)
>>>>> lm1<- lm(Reaction ~ Days, sleepstudy)
>>>>>
>>>>>
>>>>> LL<-numeric(500)
>>>>> for(i in 1:500){
>>>>> resp<-simulate(lm1)
>>>>> LL[i]<-logLik(lmer(resp[,1] ~ Days + (1|Subject), sleepstudy))
>>>>> }
>>>>>
>>>>> hist(LL)
>>>>> logLik(mm1)
>>>>> mean(LL>logLik(mm1))
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>>
>>>>>
>>>>
>>>>
>>> --
>>> Matthias Gralle, PhD
>>> Dept. Evolutionary Genetics
>>> Max Planck Institute for Evolutionary Anthropology
>>> Deutscher Platz 6
>>> 04103 Leipzig, Germany
>>> Tel +49 341 3550 519
>>> Fax +49 341 3550 555
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>
>

Dr. Achaz von Hardenberg
--------------------------------------------------------------------------------------------------------
Centro Studi Fauna Alpina - Alpine Wildlife Research Centre
Servizio Sanitario e della Ricerca Scientifica
Parco Nazionale Gran Paradiso, Degioz, 11, 11010-Valsavarenche (Ao),  
Italy

Present address:
National Centre for Statistical Ecology
School of Mathematics, Statistics and Actuarial Science,
University of Kent,  Canterbury, UK