[R-sig-ME] Interpreting results of a mixed model

[Ben Bolker]

Elizabeth Beck <elizabethbeck0 at ...> writes:

> 
> I have a question about the use of lme for hypothesis testing. I am
> interested in the effect of treatment on my response variables
> (potassium) in this case - I have about 50 blood variables total. My
> basic design is:

  Like many questions that have been appearing on r-sig-mixed-models
lately, this question isn't really specific to mixed models.  It's more
of a general question about interpreting the results of linear models.
My general advice would be to read a really good, modern, R-centric
book on statistical modeling -- I would recommend Frank Harrell's
"Regression Modeling Strategies", but it might be a bit too advanced.
(While it would be off-topic here, I would be interested in other
opinions on this subject.  I would also tentatively suggest Faraway's
"Linear Models with R" and John Fox's "Companion to Applied Regression",
but I have to admit that I don't have much first-hand experience with
those books. (You might also look for examples/
reading on the specific issues of effect modification and confounding,
e.g. biostat.mc.vanderbilt.edu/wiki/pub/Main/CourseBios312/effmod.pdf‎ --
this stuff is expressed in a slightly different set of vocabulary,
but seems highly relevant.)

> 
> >
> > $ ID: Factor w/ 36 levels  --> RANDOM EFFECT
> > $ EXPOSURE : int  -> FIXED FACTOR (REPEATED MEASURE)
> > $ TREATMENT: Factor w/ 2 levels "Control","Experimental": 
>  --> FIXED FACTOR
> > $ SEX : Factor w/ 2 levels "Female","Male": --> FIXED FACTOR
> > $ K  : num  2.2 2.3 2 3.5 ... --> RESPONSE VARIABLE
> >
> > I am unsure how to proceed to test for the effect of treatment on my
> > response as I've had several opinions leading to different results. It
> > was first suggested to me that my loaded model (in terms of fixed
> > factors) should be the final step, and I should look at the p-values
> > from the summary table which in this case yields a significant effect
> > of exposure:treatment, as well as the treatment main effect.
> >
> > M2K.lme<-lme(K~ SEX*EXPOSURE*TREATMENT, random=~1|ID, method='REML',
> > weights=varIdent(form=~1|EXPOSURE),data=biochem12)
> > > summary(M2K.lme)
> >
> > Although this makes sense, I am concerned that by over-fitting the model
> > like this (the 3-way interaction especially) I am not getting an
> > accurate result.
> >
> > I was told my another to continue with backwards selection of my fixed
> > effects which, in the case of potassium means I remove every
> > interaction and main effect leaving an empty model! While that may be
> > true biologically speaking (if none of my factors impact potassium
> > whatsoever) it makes for an awkward interpretation of my results.
> >
> > I also have the opposite problem with several variables whereby with
> > backwards selection non significant terms becomes significant - which
> > doesn't necessarily make the evidence much stronger in the biological
> > sense.
> >

  In general I prefer the "keep the full model" approach, to avoid
snooping, although then you have to be extremely careful to interpret
the main effects appropriately (marginality, least-squares means,
sum-to-zero-contrasts, blah blah blah ...).  I don't object to
mild simplification by removal of non-significant interactions, but
the fact that the interpretation changes should concern you.

 My primary advice would be to create a meaningful plot of the
data in order to understand what's actually going on.

  Ben Bolker