[R-sig-eco] Residuals in a mixed effect model

[Dixon, Philip M [STAT]]

Malin,

I would double check exactly which residuals you are getting before worrying further about heterscedasticity.
In a linear mixed model (Y = Xb + Zu + e), there are two definitions of the residual: Y-X bhat and Y - (X bhat + Z uhat).  You want the second to investigate unequal variance of the e's.

In mixed models with only random intercepts, this is not an issue because Var(Yi) is constant.  When the slope is random, Var(Yi) is not constant.  Var(Yi) depends on Xi even when Var u and Var e are constants.

I believe resid(mer object) gives you the first (Y-Xbhat), based on faint memory and a quick read of ?'mer-class'.

If you do need a stronger transformation, did you try Y^(-1/2) or Y^(-1), both of which are stronger than log(Y) in the Box-Cox family?

For any transformation, I would worry much more about linearity of the regression on the transformed scale than about heterogeneity of variances.  If the relationship isn't linear, the beta's are at best approximations and at worst irrelevant.

You may be able to model the error variance as a function of logRe or a function of something else (sample size in the study).

Finally, you may be able / willing to ignore the heterogeneity.  David Fletcher and I investigated this in a simpler meta regression problem (each study providing one observation in the meta analysis).  That came out in the March 2012 Methods in Ecology and Evolution.

Best wishes,
Philip Dixon