[R-sig-ME] LRT tests with lme4 and single random effects

[Ned Dochtermann]

Hi Jason,
The short answer is that, to my understanding, you can’t. You can’t compare the fit of an lmer model to that of an lm model, even if they’re nested, because the model deviances are calculated differently. Ben Bolker discusses this further on his GLMM wiki ( http://glmm.wikidot.com/random-effects-testing ). 

A simple likelihood ratio test with the two models won't work because they have the same numbers of parameters, ie df=0. Because of this the last line of the code actually does conduct an LRT but just does so directly with the model deviances rather than with anova().

At least this is my understanding, things may have changed.

Ned

--
Ned Dochtermann
Department of Biology
University of Nevada, Reno

ned.dochtermann at gmail.com
http://wolfweb.unr.edu/homepage/mpeacock/Ned.Dochtermann/
http://www.researcherid.com/rid/A-7146-2010
--



From: JASON M. HILL [mailto:jmh656 at psu.edu] 
Sent: Wednesday, October 19, 2011 4:21 PM
To: Ned Dochtermann
Subject: Re: [R-sig-ME] LRT tests with lme4 and single random effects

Hi Ned,
Why don't you compare the deviance of the model with and without the random effect, or conduct a likelihood ratio test between the two models?
Jason

On Wed, Oct 19, 2011 06:59 PM, "Ned Dochtermann" <ned.dochtermann at gmail.com> wrote:


Hi All,
As part of looking at some issues with repeatability from a repeated
measures mixed-model analysis I've been asked by a colleague about
statistical power in regards to the "significance" of repeatability
estimates. In other packages (SAS and ASReml) this would be
determined based
on the significance of the random effect. The problem I ran into of course
is that there isn't a ready-made way to test the significance of a random
factor in a single random effect model since there isn't a function that has
commensurate likelihoods with lme4. I know I could do this with lme and gls
but as I had been using lmer to look at the accuracy of repeatability
estimates, I'd prefer to stick with the lme4 library. I know mcmcsamp could
provide CI's around the random term's variance but that's not quite what
we're looking for.

The work around I came up with was to add in an uninformative (except to a
small degree due to chance) variable and include it as a random factor in a
separate model. The deviance for this model could then be used as a stand-in
for a model omitting the random factor.

My code to do this was:
	icc<-0.3; n<-100; reps<-2
	ind.0<-rnorm(n,0,sqrt(icc)
	ind.e<-rnorm(n*reps,0,sqrt(1-icc)
	ind<-gl(n,1)

ind.data<-data.frame(cbind(sample(c(1:n),n*reps,T),
	rep(seq(n),reps),
	matrix(ind.0[ind]+ind.e,ncol=1)
	col.names<-c("rnd","ind","y"); names(ind.data)<-col.names
	
	mm1<-lmer(y~1+(1|ind),data=ind.data)
	mmN<-lmer(y~1+(1|rnd),data=ind.data)

	mm1.dev<-summary(mm1)@deviance[1]
	mmN.dev<-summary(mmN)@deviance[1]
	(lrt.p<-1-pchisq(abs(mm1.dev-mmN.dev),df=1)
	
If lrt.p is less than alpha this then suggests a "significant" random
effect. Due to the issues with p-values I suppose lrt.p should also be
divided by two. It seems like this should work, although since 1|rnd will by
chance reduce the deviance some small amount it might be conservative. Is
this a completely idiotic approach due to me missing something obvious to
statisticians that was not obvious to me as a user? 

Sorry if this has come up, I did search but didn't find anything that quite
fit what I was looking for.
Ned

--
Ned Dochtermann
Department of Biology
University of Nevada, Reno

ned.dochtermann at gmail.com
http://wolfweb.unr.edu/homepage/mpeacock/Ned.Dochtermann/
http://www.researcherid.com/rid/A-7146-2010
--

_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models


'Nothing in biology makes sense, except in the light of evolution'
--Theodosius Dobzhansky--

Jason Hill
http://www.coopunits.org/Pennsylvania/People/Jason_Hill/index.html
PA Cooperative Fish and Wildlife Research Unit
221 Forest Resources Building
University Park, PA 16802-4705
Office: 814-865-0772
Fax: 814-863-4710
Ecology Program - PhD Candidate
Pennsylvania State University
School of Forest Resources