[R-sig-ME] Replicating type III anova tests for glmer/GLMM

Tue Feb 23 13:54:05 CET 2016

In my experience, car::Anova is slightly less conservative (as Wald
tests are known to be somewhat anti-conservative).

Are you using Type-III tests for everything? The differences between
Type-II and Type-III can actually make a big difference in terms of
which predictors are significant.

Speaking of Type-III -- although it's the default in some popular
commercial packages, Type-II (marginal tests) is actually the type that
makes the most sense in terms of statistical interpretation and
hypotheses tested. But that's a topic for another time ....

Best,
Phillip

On 23/02/16 22:41, Francesco Romano wrote:
> Thanks to Henrik and Phillip for the quick reply.
> Your suggestions have been helpful in making progress.
> 
> On the one hand Henrik is right about
> reporting coefficients and standard errors when
> there are only two levels for the each predictor. This is
> consistent with two of the sources I mentioned so far.
> I infer that the authors reported directly from the summary(m1)
> after use of the mixed function (not car::Anova which yields chi
> square tests).
> 
> On the other hand, I don't understand how Cai et al. (2012) p.842,
> "combined analysis experiments 1 and 2", reported the main effect
> of a factor with 4 levels via a single estimate, SE, z, p coefficient.
> How did they obtain this and is this the right way?
> 
> Finally, after running analysis both ways, I get slightly different
> p-values, with the car::Anova method being more conservative
> (it yields less significant predictors). Is this normal?
> 
> Frank
> 
> 
> 
> On Tue, Feb 23, 2016 at 10:51 AM, Phillip Alday <Phillip.Alday at unisa.edu.au>
> wrote:
> 
>> lme4:anova() is not the same thing as car::Anova()!
>>
>> A quick R note that might have avoided the confusion:
>> The :: syntax in R refers to scope, so you can specify a function
>> unambiguously via package::function.name(). Moreover, R is case
>> sensitive, so Anova() and anova() are generally different things.
>>
>> Henrik's message (posted to the list so if you don't suscribe, you need
>> to look here:
>>
>> https://mailman.stat.ethz.ch/pipermail/r-sig-mixed-models/2016q1/024465.html
>> ) describes how to do this with either his afex package (for
>> likelihood-ratio tests) or John Fox's car package (for analysis of
>> deviance / Wald tests).
>>
>> If you just want to perform likelihood-ratio tests in lme4, then you
>> should look at the drop1() function or you can use anova(reduced.model,
>> full.model). Henrik also does a nice job summarizing some of the issues
>> here, so I won't repeat them.
>>
>> One final note: not everything that holds for normal LMM holds for GLMM
>> -- GLMM tends to be much more complicated. :-(
>>
>> Best,
>> Phillip
>>
>> On 23/02/16 20:03, Francesco Romano wrote:
>>> Yes. An ANOVA with my final bglmer model yields:
>>>
>>>> anova(recallmodel4x6a)
>>>
>>> Analysis of Variance Table
>>>
>>>                    Df Sum Sq Mean Sq F value
>>> syntax12            1 1.7670  1.7670  1.7670
>>> animacy12           1 3.4036  3.4036  3.4036
>>> group123            2 5.7213  2.8607  2.8607
>>> animacy12:group123  2 4.5546  2.2773  2.2773
>>> syntax12:group123   2 8.1732  4.0866  4.0866
>>>
>>> which is counterintuitively not what the authors of the papers
>>> apparently used to generate coefficients to report their main effects
>>> and interactions. It looks to me more like ML fitting. Elsewhere,
>>> and more typically, main effects and interactions are obtained by
>>> comparing a
>>>
>>> model with the main fixed effect to a model without the
>>>
>>> main fixed effect in terms of log-likelihood ratio tests
>>>
>>> (Raffray et al., 2013, http://dx.doi.org/10.1016/j.jml.2013.09.004,
>> p.6).
>>>
>>>
>>> I understand obtaining p-values from a summary
>>> of linear mixed models fit by lmer is a contentious issue
>>>
>>> https://stat.ethz.ch/pipermail/r-help/2006-May/094765.html
>>>
>>> but I guess I might be missing something here.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Feb 23, 2016 at 2:21 AM, Phillip Alday
>>> <Phillip.Alday at unisa.edu.au <mailto:Phillip.Alday at unisa.edu.au>> wrote:
>>>
>>>     Have you looked at car::Anova() ?
>>>
>>>     Best,
>>>     Phillip
>>>
>>>     [forgot to cc the list]
>>>
>>>     > On 23 Feb 2016, at 11:42, Francesco Romano <
>> francescobryanromano at gmail.com
>>>     <mailto:francescobryanromano at gmail.com>> wrote:
>>>     >
>>>     > Dear all,
>>>     >
>>>     > I'm trying to report my analysis replicating the method in the
>>>     following
>>>     > papers:
>>>     >
>>>     > Cai, Pickering, and Branigan (2012). Mapping concepts to syntax:
>>>     Evidence
>>>     > from structural priming in Mandarin Chinese. Journal of Memory and
>>>     Language 66
>>>     > (2012) 833–849 <tel:%282012%29%20833%E2%80%93849>. (looking at pg.
>>>     842, "Combined analysis of Experiments 1
>>>     > and 2" section)
>>>     >
>>>     > Filiaci, Sorace, and Carreiras (2013). Anaphoric biases of null
>>>     and overt
>>>     > subjects in Italian and Spanish: a cross-linguistic comparison.
>>>     Language,
>>>     > Cognition, and Neuroscience  DOI:10.1080/01690965.2013.801502
>>>     (looking at
>>>     > pg.11, first two paragraphs)
>>>     >
>>>     > This is because I have a glmer model with three fixed effects, two
>>>     random
>>>     > intercepts modeling a binary outcome, exactly as in the articles
>>>     mentioned.
>>>     >
>>>     > The difficulty I'm finding is with locating information on commands
>>>     > generating coefficients, SE, z, and p values (e.g. maximum
>> likelihood
>>>     > (Laplace Approximation)) to report main effects and interactions
>>>     with the
>>>     > anova or afex:mixed commands, following application of effect
>>>     coding. I
>>>     > have looked in several places, including Ben Bolker's FAQ
>>>     > http://glmm.wikidot.com/faq and past posts on the topic in this
>> r-sig.
>>>     > Although there appears to be a plethora of material for lmer, I
>>>     can't seem
>>>     > to locate anything in the right direction for glmer.
>>>     >
>>>     > Many thanks for any help.
>>>     >
>>>     >
>>>     >
>>>     >
>>>     > --
>>>     > Frank Romano Ph.D.
>>>     >
>>>     > *LinkedIn*
>>>     > https://it.linkedin.com/pub/francesco-bryan-romano/33/1/162
>>>     >
>>>     > *Academia.edu*
>>>     > https://sheffield.academia.edu/FrancescoRomano
>>>     >
>>>     >       [[alternative HTML version deleted]]
>>>     >
>>>     > _______________________________________________
>>>     > R-sig-mixed-models at r-project.org
>>>     <mailto:R-sig-mixed-models at r-project.org> mailing list
>>>     > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>>
>>>
>>>
>>> --
>>> Frank Romano Ph.D.
>>>
>>> Tel. +39 3911639149
>>>
>>> /LinkedIn/
>>> https://it.linkedin.com/pub/francesco-bryan-romano/33/1/162
>>>
>>> /Academia.edu/
>>> https://sheffield.academia.edu/FrancescoRomano
>>
> 
> 
> 