[R-sig-ME] New alpha test release (0.999375-5) with mcmcsamp for linear mixed models

[Douglas Bates]

Yesterday I uploaded the files for alpha test release 0.999375-5 to
R-forge.R-project.org

Packages should be built and tested overnight.  I understand from
announcements by the R-forge administration team that binary packages
for Mac OS X should now be available through

install.packages("lme4", repos = "http://r-forge.r-project.org")

although I haven't tested this on a Mac myself.

The "show stopper" for the release of version 1.0.0 of the lme4
package has been getting mcmcsamp running again.  I have made dramatic
changes to the underlying structure of the objects representing
mixed-effects models and I wanted to be able to have mcmcsamp running
in this new formulation before an official release of a new version.

The good news is that I have mcmcsamp running for linear mixed models
and I think the samples look as they should.  I would, however, very
much appreciate others testing this function and checking if the
results are consistent with earlier implementations.  Because they are
generated by a pseudo-random mechanism in a different algorithm they
will not be identical to earlier results but the distribution should
be similar.

The bad news is that I changed the representation of the result of
mcmcsamp and this will have ramifications for code maintained by
others.  Harald Baayen and Andrew Gelman's group in particular may
want to check the new representation of the results from MCMCsamp.
The merMCMC class is an S4 class whose slots are described in the
documentation

class?merMCMC

The sample from the distribution of the fixed-effects parameters are a
matrix in a slot called "fixef".  Right now the iterations correspond
to columns - it may be more natural to transpose that matrix when
examining it.  The first sample is the fitted value of the fixed
effects.  The "sigma" slot is the corresponding sample of the common
scale parameter.  The variance-covariance of the random effects is
derived from the "sigma" slot and the "ST" slot.

To decode the representation in the ST slot you need the "nc" (number
of columns) slot which is an integer vector giving the number of
columns in the random effects matrices corresponding to each random
effects term in the model. I write those as q_i, i = 1, ..., k.  A
column of the matrix in the ST slot is of length np = \sum_{i=1}^k
q_i(q_i + 1)/2.  When divided into these subvectors, the first q_i
elements of a subvector are the non-negative scale factors from the
diagonal of the diagonal matrix S_i and the remaining elements are the
elements below the diagonal in the unit lower triangular matrix T_i.
S_i, T_i and sigma generate the variance-covariance matrix for the
random effects corresponding to the ith term as

sigma^2 * T_i %*% S_i %*% S_i %*% t(T_i)

See http://www.stat.wisc.edu/~bates/2008-02-22-Emory.pdf for a
worked-out example of this calculation.

I will create the support functions such as HPDinterval and methods
for generics like xyplot and densityplot for this class.  First I want
to get mcmcsamp working for GLMMs and NLMMs.