[R-sig-ME] [Fwd: Re: Wald F tests]

[Douglas Bates]

On Tue, Oct 7, 2008 at 4:51 PM, Ben Bolker <bolker at ufl.edu> wrote:

>  But ... LRTs are non-recommended (anticonservative) for
> comparing fixed effects of LMMs hence (presumably) for
> GLMMs, unless sample size (# blocks/"residual" total sample
> size) is large, no?

> I just got through telling readers of
> a forthcoming TREE (Trends in Ecology and Evolution) article
> that they should use Wald Z, chi^2, t, or F (depending on
> whether testing a single or multiple parameters, and whether
> there is overdispersion or not), in preference to LRTs,
> for testing fixed effects ... ?  Or do you consider LRT
> better than Wald in this case (in which case as far as
> we know _nothing_ works very well for GLMMs, and I might
> just start to cry ...)  Or perhaps I have to get busy
> running some simulations ...

My reasoning, based on my experiences with nonlinear regression models
and other nonlinear models, is that a test that involves fitting the
alternative model and the null model then comparing the quality of the
fit will give more realistic results than a test that only involves
fitting the alternative model and using that fit to extrapolate to
what the null model fit should be like.

We will always use approximations in statistics but as we get more
powerful computing facilities some of the approximations that we
needed to use in the past can be avoided.  I view Wald tests as an
approximation to the quantity that we want to use to compare models,
which is some measure of the comparative fit.  The likelihood ratio or
the change in the deviance seems to be a reasonable way of comparing
the fits of two nested models.  There may be problems with calibrating
that quantity (i.e. converting it to a p-value) in which case we may
want to use a bootstrap or some other simulation-based method like
MCMC.  However, I don't think this difficulty would cause me to say
that it is better to use an approximation to the model fit under the
null hypothesis than to go ahead and fit it.

>  Where would _you_ go to find advice on inference
> (as opposed to estimation) on estimated GLMM parameters?

I'm not sure.  As I once said to Martin, my research involves far too
much "re" and far too little "search".  Probably because of laziness I
tend to try to reason things out instead of conducting literature
reviews.

>  cheers
>   Ben Bolker
>
> Douglas Bates wrote:
>> If I were using glmer to fit a generalized linear mixed model I would
>> use likelihood ratio tests rather than Wald tests.  That is, I would
>> fit a model including a particular term then fit it again without that
>> term and calculate the difference in the deviance values, comparing
>> that to a chi-square.
>>
>> I'm not sure how one would do this using the results from glmmPQL.
>>
>> On Fri, Oct 3, 2008 at 3:37 PM, Ben Bolker <bolker at ufl.edu> wrote:
>>>  [forwarding to R-sig-mixed, where it is likely to get more
>>> responses]
>>>
>>> Mark Fowler wrote:
>>>
>>> Hello,
>>>        Might anyone know how to conduct Wald-type F-tests of the fixed
>>> effects estimated by glmmPQL? I see this implemented in SAS (GLIMMIX),
>>> and have seen it recommended in user group discussions, but haven't come
>>> across any code to accomplish it. I understand the anova function treats
>>> a glmmPQL fit as an lme fit, with the test assumptions based on maximum
>>> likelihood, which is inappropriate for PQL. I'm using S-Plus 7. I also
>>> have R 2.7 and S-Plus 8 if necessary.
>>>
>>> _______________________________________________
>>> R-sig-mixed-models at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>