[R-sig-ME] Follow-up on lme4

[Douglas Bates]

I thought that this reply may be of interest to the members of the
list.  Because I did not ask permission of the questioner before
sending a copy to the list, I have removed the questioner's name.

The short answer to your question is that the lme4 package is a work
in progress but it will converge to a 1.0 release "real soon".

By the 1.0 release there will not be an lmer2 function.  The
model-fitting functions will be lmer, for linear mixed models, glmer,
for generalized linear mixed models, and nlmer, for nonlinear mixed
models.  (The official pronunciations are "Elmer", "glimmer" and
"Nellmer", for those who care about such things.)

It is not that the lmer2 representation will go away; it will become
the lmer representation in the 1.0 version and will also be the
"linear mixed-effects engine" on which glmer and nlmer are built.  The
current lmer is the one that will go away.

The model specification for the 1.0 lmer, glmer and nlmer will be the
same as for the current lmer and nlmer.  (In the current version there
is not a separate function for generalized linear mixed models, you
just call lmer with a non-default "family" argument.  This behavior
will be "grandfather'ed" although all lmer will end up doing with a
glmm specification is turn around and call glmer.)

The internal representation of the fitted models will *not* be the
same as in the current version of the lme4 package.  This is why it
would be a very good idea to save the data and the model
specifications from current lmer fits.  I will make preview versions
of the package available and I would greatly appreciate if those of
you who have published results from lmer fits could check that the 1.0
version of lme4 gives a fit that is at least as good as the one you
reported.  If not, please inform me and I will do whatever I can to
ensure suitable behavior on models fit previously before I release the
1.0 version of lme4.

The reason that I have not already switched the package to the lmer2
representation is because I was having difficulty getting two pieces
of code, generalized linear mixed models and mcmcsamp, to work
properly in the new formulation.  I use two scales for the random
effects, the original scale and a transformed "orthogonal" scale, in
the lmer2 representation and at some point I must have thought of the
form of the calculation in one scale but used the other.

Eventually I decided to bite the bullet and write out the gory details
of the computational methods so I could be sure that I understood all
the steps and hadn't missed any steps when writing the code. As my
friend Steve Ziskind from graduate school said, "Every once in a while
you need to roll up your sleeves and write it out."  I expect that by
the time I am finished writing it all out and honing the steps in the
presentation, the actual writing of the code will be straightforward.
I know that the code to create the representation is very clean
relative to previous versions lmer (and certainly relative to lme and
nlme).  The development version of the code, currently broken, is at

https://svn.r-project.org/R-packages/branches/gappy-lmer/

Those of you who liked the scenes in the Frankenstein movies before
the Doctor proclaimed "It's alive!" can satisfy your curiosity there.
There's a reason why this representation can be considered to have
gaps but it would take too long to explain.

The "version du jour" of the paper explaining the theory and the
computational methods is available at

http://www.stat.wisc.edu/~bates/Mixed-Model-Theory.pdf

On 6/26/07, an lme4 user wrote:

>  Is lmer2 an integrated component of the lme4 package, or does it have to be
> downloaded separately ?

It is part of the current lme4 package.  Eventually it will become lmer.

>  Also -- as I understand it -- lme4 is intended to eventually expand the
> capabilities and performance of your older nlme library ?

Functions in the lme4 package extend the capabilities of lme and nlme
in some directions - specifically the ability to fit models with
crossed or partially crossed random effects, to fit generalized linear
mixed models and to fit nonlinear mixed models using the Laplace
approximation to the log-likelihood.  The model-fitting functions also
handle the boundary cases, corresponding to variance components of
zero or more general, but degenerate, forms of the
variance-covariance.

However, lme4 does not include some capabilities from the nlme package
- specifically the ability to model correlation structures or variance
structures in the responses in addition to those induced by the random
effects.