[R-sig-ME] need help with predicted values from lmer with newdata

[Joshua Wiley]

On Sat, Jun 23, 2012 at 4:27 AM, Ben Bolker <bbolker at gmail.com> wrote:
[snip]

>  In addition, any advice on how to improve ?predict.merMod ("I can't
> tell what those numbers are. Can't be sure from ?predict.merMod") would
> be welcome ...

Far from perfect, but what about something like this?  I do not know
about others, but I often prefer to just see the formulae.  In these
models, the questions I always have are
1) are the random effects being included (nicely shown by Zu)
2) not relevant for the lmer() case, but for glmer, are the predicted
values conditional, unconditional, marginal

when \code{predict} is called on a model object without a new dataset,
it returns the fitted values.  By default, these include the random
effects.  That is, \eqn{\hat{Y} = \mathbf{Xb} + \mathbf{Zu}}{Yhat = Zb
+ Zu}.  Where \eqn{\mathbf{X}}{X} is the fixed-effects design matrix,
\eqn{\mathbf{b}}{b} is the vector of fixed paramter estimates,
\eqn{\mathbf{Z}}{Z} is the random-effects design matrix, and
\eqn{\mathbf{u}}{u} is the vector of random parameter estimates. If
you set \code{REform=NA}, then you will get the unconditional fitted
values.  That is, \eqn{\hat{Y} = \mathbf{Xb}}{Yhat = Xb}.

## FIXME: perhaps add information about link functions

To calculate predicted values from a new dataset (i.e., when
\code{newdata} is not \code{NULL}), the model matrix,
\eqn{\mathbf{X}}{X} is created from \code{newdata}, using the
contrasts from the original model if there are any categorical
(factor) variables.  The fixed portion of the predicted values are
calculated as \eqn{\hat{Y} = \mathbf{Xb}}{Yhat = Xb}.  If \code{REform
= NULL} (the default), then all random effects are used.  If
\code{REform = NA} then no random effects are used, otherwise,
\code{REform} can be a formula specifying which random effects to
include.  These options can be thought of as setting a diagonal
selection matrix, \eqn{\mathbf{D}}{D}.  In the default case, it is the
identity matrix, in the second case, the diagonal is all 0s, and in
the formula case the diagonal contains a combination of 0s and 1s.
Then the final predicted values are \eqn{\hat{Y} = \mathbf{Xb} +
\mathbf{Z}(\mathbf{Du})}{Yhat = Xb + Z(Du)}.



>
>  Ben Bolker
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models



-- 
Joshua Wiley
Ph.D. Student, Health Psychology
Programmer Analyst II, Statistical Consulting Group
University of California, Los Angeles
https://joshuawiley.com/