[R-sig-ME] Mixed model predictions using sim

Thu Dec 10 11:01:35 CET 2015

Hi Vince

Thanks for that.  I think I should expand on my question:

A ‘snip’ from a 20,000 simulation run is attached.

The intercept values quantiles for the intercepts (52.26827...67.6602) are repeated in the Predicted values (LowerCI, Middle and UpperCI) for ‘0’ units of ‘Open’, as we’ve noted.  Taking 60 units as an example, how do you determine the UpperCI?  It could be 67.66602+(60*0.016412019) = 68.6507 which is slightly (and trivially in this example) different from the UpperCI given in here.  In my real example the difference is larger (and meaningful).  My real example is based on a more complex model (multiple predictors – all bar one of which are centred (and set at zero in the predictions) so don’t enter the manual calculation as used above).  The difference does not seem to be related to the number of simulations (there are still differences, of the same-magnitude, with 200,000 simulations).   Could the difference be attributable to the line:

for(i in 1:nsim) fitmat[,i]=(MLexamp.9.Pred.Matrix%*%MLexamp.9.sim at fixef[i,])

Does this line indicate that model predictions are based on the fixed-effects only from the simulations?  Basically, I’m still uncertain about how the Credible Intervals, for individual parameters in the model, relate to the Confidence Intervals and why there is an apparent difference.

Many thanks

Tom.

From: Vincent Dorie [mailto:vjd4 at nyu.edu]
Sent: 09 December 2015 21:06
To: Tom Wilding
Cc: R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org>
Subject: Re: [R-sig-ME] Mixed model predictions using sim

Hi Tom,

If I'm following you correctly, the difference is due to the inclusion of uncertainty in the slope term which is not present in the term for open != 0. The quantiles are actually identical if you omit the signif call.

A more accurate representation of uncertain for new observations might be to use the posterior predictive distribution. This can be obtained by adding some rnorm using the sample of sigma (assuming that you were interested in observations in a new group).

Vince

On Wed, Dec 9, 2015 at 5:18 AM, Tom Wilding <Tom.Wilding at sams.ac.uk<mailto:Tom.Wilding at sams.ac.uk>> wrote:
Dear Mailing list

I'm using lme4 to conduct mixed models and then, in accordance with several authors I'm using the library 'arm' to simulate from that model to generate confidence intervals for my parameter estimates (see Korner-Nievergelt, F. et al ., 2015. Bayesian data analysis in ecology using linear models with R, BUGS and Stan. Elsevier).

I would like to know more about the relationship between the Credible Intervals (CrI) and the Confidence Intervals (2.5,50 and 97.5% quantiles) generated from model predictions.  I'm particularly interested to know why the Credible Intervals (given in 'CrI.MLexamp.9' below), for values of the intercept (i.e. when the predictor is zero) are reflected very accurately in the quantiles from the Predictions ('MLexamp.9.Pred' as below) yet there is a discrepancy between values predicted by the CrI and the 95% CI quantiles from the Predictions at, e.g., 60 units of 'open' in the example below.  How does one move between the CrI to the model predictions?  Why the difference?  (In my real example, the discrepancy is more meaningful).

The data (thanks) I've used in this example is from:

http://jaredknowles.com/journal/2013/11/25/getting-started-with-mixed-effect-models-in-r

Any pointers would be much appreciated.

Thanks

Tom
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The following script might take 30 seconds to run...

library(lme4)
library(arm)
lmm.data=read.table("http://www.unt.edu/rss/class/Jon/R_SC/Module9/lmm.data.txt",
                       header = TRUE, sep = ",", na.strings = "NA", dec = ".", strip.white = TRUE)
MLexamp.9=lmer(extro ~ open  + (1 + open | school/class),
                  data = lmm.data)
#summary(MLexamp.9)
nsim=20000
MLexamp.9.sim=sim (MLexamp.9,n.sim=nsim); #
colnames(MLexamp.9.sim at fixef)=names(fixef(MLexamp.9))<mailto:MLexamp.9.sim at fixef<mailto:MLexamp.9.sim at fixef>)=names(fixef(MLexamp.9))>

#generate CrIs.
CrI.MLexamp.9=as.data.frame(signif(apply(MLexamp.9.sim at fixef,2,quantile,prob=c(0.025,0.5,0.975)),5))<mailto:MLexamp.9.sim at fixef,2,quantile,prob=c(0.025,0.5,0.975)),5))>

MLexamp.9.Pred=data.frame(open=c(0,20,40,60))#realistic values and zero.
MLexamp.9.Pred.Matrix=model.matrix(~open,data=MLexamp.9.Pred)
MLexamp.9.Pred$fit=(MLexamp.9.Pred.Matrix%*%fixef(MLexamp.9))#

fitmat=matrix(nrow=nrow(MLexamp.9.Pred),ncol=nsim)#
for(i in 1:nsim) fitmat[,i]=(MLexamp.9.Pred.Matrix%*%MLexamp.9.sim at fixef[i,])#<mailto:MLexamp.9.Pred.Matrix%25*%25MLexamp.9.sim at fixef[i,])>

MLexamp.9.Pred$LowerCI=apply(fitmat,1,quantile,prob=0.025)
MLexamp.9.Pred$Middle=apply(fitmat,1,quantile,prob=0.500)
MLexamp.9.Pred$UpperCI=apply(fitmat,1,quantile,prob=0.975)
CrI.MLexamp.9
MLexamp.9.Pred
The Scottish Association for Marine Science (SAMS) is registered in Scotland as a Company Limited by Guarantee (SC009292) and is a registered charity (9206). SAMS has two actively trading wholly owned subsidiary companies: SAMS Research Services Ltd (SC224404) and SAMS Ltd (SC306912). All Companies in the group are registered in Scotland and share a registered office at Scottish Marine Institute, Oban Argyll PA37 1QA. The content of this message may contain personal views which are not the views of SAMS unless specifically stated. Please note that all email traffic is monitored for purposes of security and spam filtering. As such individual emails may be examined in more detail.

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-models at r-project.org<mailto:R-sig-mixed-models at r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

The Scottish Association for Marine Science (SAMS) is registered in Scotland as a Company Limited by Guarantee (SC009292) and is a registered charity (9206). SAMS has two actively trading wholly owned subsidiary companies: SAMS Research Services Ltd (SC224404) and SAMS Ltd (SC306912). All Companies in the group are registered in Scotland and share a registered office at Scottish Marine Institute, Oban Argyll PA37 1QA. The content of this message may contain personal views which are not the views of SAMS unless specifically stated. Please note that all email traffic is monitored for purposes of security and spam filtering. As such individual emails may be examined in more detail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Mixed Model CrI and CI examplar.PNG
Type: image/png
Size: 14666 bytes
Desc: Mixed Model CrI and CI examplar.PNG
URL: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/attachments/20151210/b27cc889/attachment-0001.png>