[R-sig-ME] cross-validation of linear mixed effect models

[Mike Lawrence]

One thought is that according to Fang (2011,
http://www.jds-online.com/file_download/278/JDS-652a.pdf), AIC for a
mixed effects model is asymptotically equivalent to
leave-one-cluster-out cross-validation, so possibly you already have a
metric of cross-validated prediction accuracy in your AIC scores. Now
that I think of it though, I see Fang makes a distinction between
marginal vs conditional AIC, and I'm not sure though which is
implemented when you submit an lmer model to the AIC() function in R.
Also, I'm not sure if the use of REML in fitting the model affects the
equivalence asserted by Fang; certainly I've been advised on this list
before that if I want to compare nested models on AIC scores, I needed
to ensure that REML=F in the fitted models.


On Sun, Feb 5, 2012 at 2:41 PM, Rachel Cohen <stat.list at yahoo.co.uk> wrote:
> Hi, I am a PhD student who is using mixed effect models (and R) for the first time so apologies if my question is a bit basic.  I also haven't used this forum much so if this topic has already been covered in depth in the past and there is an easy answer out there that I've missed then additional apologies!
>
> I am using a linear mixed model (structure as below) to predict total tree biomass (log.mass) using tree diameter (dbh) and height as explanatory variables, allowing the intercept and the slope of height to vary by my grouping factor (species_site).
>
> Model:
>
> lmer((log.mass)~centre.log.dbh+centre.log.height+(1+centre.log.height|species_site),data=data3,REML=T)
>
> This model structure was chosen as 'the best' primarily on the basis of AIC value.   Residual plots look fairly OK.
>
> I would now like to check the predictive performance of my model (by cross-validation?) and wonder how to go about this in the context of a linear mixed effect model?.  Is there a package/function in R which deals with this?
>
> Any advice on how to proceed would be greatly appreciated!
>
> Regards,
>
> Rachel Cohen
>        [[alternative HTML version deleted]]
>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>