[R-sig-ME] Logistisc regression (lmer) fitted by the Laplaceapproximation- references?

[Douglas Bates]

On Wed, Jul 23, 2008 at 7:46 AM, Doran, Harold <HDoran at air.org> wrote:
> Do you mean limitations of the laplace approximation or limits of a glmm
> fit by lmer? If it is the former, one possible argument is that lmer
> uses a second-order taylor series expansion. One might claim that going
> out further in the taylor series is needed to get good approximations of
> the integral. I, however, may not be one of those people.

> I know for example that the HLM software package uses a 6 order taylor
> series. Just last week, I had to replicate some work. I used lmer and
> HLM to compare output. The R output and the HLM output matched exactly
> to the 3th decimal place for the BLUPS and the fixed effects

I'm not sure what HLM does in terms of a 6th order Taylor series
expansion.  Before you can make sense of terms like "second-order
Taylor series expansion" versus "6th order" you should make clear what
is being approximated and, more importantly, where.

It has taken me a long time to get straight in my mind exactly what is
being approximated, and I am the co-author of papers from more than 10
years ago about this approximation.  These days the phrase that I use
is that we want to approximate the integral of "the unscaled
conditional density of the random effects given the observed data".
(See slides 61 to 65 in
http://www.stat.wisc.edu/~bates/IMPS2008/lme4D.pdf).

Given the observed data and values of the parameters, the procedure in
glmer is to determine the conditional modes of the random effects
(i.e. the values that maximize the conditional density).  The quantity
that determines the Laplace approximation used in glmer is the
second-order Taylor series approximation of the logarithm of the
conditional density at the conditional modes.

I did read the paper that Steve Raudenbush wrote about the 6th order
Taylor series approximation used in HLM but I have forgotten the
details.  At this point I would need to rephrase it in these terms
before I could compare the method to that used in glmer.  I believe
the method requires a single, simple, scalar random-effects term in
the model.  That is, it cannot be used with multiple grouping factors
or with vector-valued random effects.  Most higher-order
approximations require the simple model form because that is the only
way that you can take the integral with respect to the vector and
split it into scalar integrals.

I don't think it is betraying a confidence to say that Steve
Raudenbush told me that after he and his co-author had done
considerable work deriving the 6th order Laplacian approximation they
compared it to adaptive Gauss-Hermite quadrature (AGQ) and found that
AGQ with a small number of quadrature points (3 to 5) is more
accurate.  Bin Dai is implementing AGS in lme4 for models with a
single grouping factor.  These include  models with a single, scalar-
or vector-valued random-effects term and models with multiple
random-effects terms based on the same grouping factor.

> Even using lmer for IRT (rasch) work gives the same point esimtates for
> the items (when they are treated as fixed effects) as I get in other IRT
> software packages that use estimating algorithms other than laplace.

But the thing that lme4 can do that I don't think much other software
can do is fit an IRT model with random effects for the items and for
the subjects.  The sorts of extensions described in "Explanatory Item
Response Models", edited by Paul De Boeck and Mark Wilson (Springer,
2004) can also be fit in lme4.
>
>
>> -----Original Message-----
>> From: r-sig-mixed-models-bounces at r-project.org
>> [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf
>> Of R.S. Cotter
>> Sent: Wednesday, July 23, 2008 6:10 AM
>> To: r-sig-mixed-models at r-project.org
>> Subject: [R-sig-ME] Logistisc regression (lmer) fitted by the
>> Laplaceapproximation- references?
>>
>> Dear all,
>>
>> I have sucessfully run Logistic regression by using
>> generalized linear mixed-effects model (lmer) fitted by the
>> Laplace approximation (lme4 package).
>>
>> Is there any limits that I should aware of by use of this
>> model? I haven't found references for this model, could
>> somone provide me with a reference from a article/book?
>>
>> My response is Yes or No and explanatory variables is
>> categories (A,B,C and D), and random effect (ID, number of groups 7).
>>
>> Regards RS
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>