[R-sig-ME] mcmcsamp and the prior on the variance components

[Douglas Bates]

In a previous message I mentioned that Steven Novick of
GlaxoSmithKline had written to me regarding peculiar behavior in some
chains generated by mcmcsamp in the lme4 package.  I enclose a script
that he provided to reproduce the problem.  I modified it slightly to
only try the BRugs model if the package exists.  I never use Windows
so I don't have access to BRugs.

If you run the script and check the plots you will see that the
estimate of the logarithm of the batch standard deviation takes
excursions when the number of batches in the sample is small.  These
excursions correspond to estimates that are very small, effectively
zero.  The prior being used is an improper, non-informative prior that
corresponds to a locally uniform prior on the logarithm of the
variance (or, equivalently, the logarithm of the standard deviation).
The end result is that the posterior distribution of the logarithms of
the standard deviations is proportional to the likelihoods.  It will
always be true that the likelihood, as a function of the logarithm of
the batch variance, reaches a plateau at large negative values,
corresponding to batch variances very close to zero.  When you have a
small number of batches it is possible for the chain to jump onto that
plateau and then it takes a long time for the chain to get off the
plateau.

I illustrate the plateau with a R script and the resulting plot.  This
script requires the development version of the lme4 package.

Steve's colleague John Peterson directed us to a paper by James Hobert
and George Casella in JASA, 1996 on "The Effect of Improper Priors on
Gibbs Sampling in Hierarchical Linear Models" that could be of
interest regarding this issue.

The big question is what to do about the plateau effect.  It results
in an improper posterior density so I can appreciate the argument that
the improper prior is not suitable.  I view improper priors as
reasonable if they produce proper posterior densities but that won't be
the case here.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mcmc_ex_R.txt
URL: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/attachments/20071018/3eea0d29/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mcmc_ex1_Rout.txt
URL: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/attachments/20071018/3eea0d29/attachment-0001.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: profiled.pdf
Type: application/pdf
Size: 11638 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/attachments/20071018/3eea0d29/attachment.pdf>