[R-sig-ME] Reporting main effects in glmer

[Ben Bolker]

Francesco <fbromano at ...> writes:

>

  I think that your basic problem here is that you are
seeing _complete separation_ in your groups; you mention that
"...   one group (the reference level group in this analysis) has
no 0s and all 1s."  This gives rise to the _Hauck-Donner_ effect,
discussed by Venables and Ripley (MASS: http://tinyurl.com/hauckdonner)
and easily Googlable, where the Wald estimates of the standard
errors and p-values (which are what's reported by summary()) are
bogus.  

Using group C as your reference level is particularly problematic: by
reporting a baseline log-odds of -95 for the intercept, glmer is
essentially getting as close to -Inf as it can before giving up.  All
other comparisons, which are made with reference to the baseline
levels, will be similarly enormous.  (For reference, any estimate with
an absolute value of >10 corresponds to proportional differences in
odds of more than 22,000, and probably indicates complete separation,
or some other issue.)  You have log-odds differences of 90 between C
and A and 91 between C and B, which are bound to be significant with
any sensible data set.

Some of your choices:

(1) use confint() to get likelihood profile confidence intervals,
not subject to the Hauck-Donner effect (but this won't give you
p-values;
(2) use bglmer from the blme package to impose weak priors on
the fixed-effect parameters;
(3) simply report the overall effect.

You should probably do some reading on complete separation
and Hauck-Donner ...

 cheers
    Ben Bolker

[snip]

> 'Correct' is a binary DV and X is a between-subjects, categorical IV 
> factor with three levels (A, B, and C).
> 
> The original call was:
> 
>  >object<-glmer(Correct~1+X+(1|Part)+(1|Item), family=binomial, data=data)
>  >summary(object)
> 
> Generalized linear mixed model fit by maximum likelihood ['glmerMod']
>   Family: binomial ( logit )
> Formula: Correct ~ 1 + X + (1 | Part) + (1 | Item)
>     Data: .....
> 
>        AIC       BIC    logLik  deviance
>   218.9053  242.0455 -104.4526  208.9053
> 
> Random effects:
>   Groups Name        Variance Std.Dev.
>   Part   (Intercept) 9.0129 3.0021 
>   Item   (Intercept) 0.7356   0.8577
> Number of obs: 756, groups: Part, 45; Item, 18
> 
> Fixed effects:
>                Estimate Std. Error z value Pr(>|z|)
> (Intercept)     -95.14 4122464.75       0        1
> A             89.88 4122464.75       0        1
> B             90.75 4122464.75       0        1
> 
> Correlation of Fixed Effects:
>       (Intr)   A
> A -1.000
> B -1.000  1.000
> 
 [snip] ...

> >     where Part and Item are random effects for participant and items.
> >     Basically the three groups are very similar in their response,
> >     where one group (the reference level group in this analysis) has
> >     no 0s and all 1s. Releveling generates a Pr(>|z|) of .54 for the
> >     difference between group B and C so I believe the model is ok.
> >
> >
> >
> >     In this scenario I would not expect A to have a main effect on the
> >     model but when I compare it to the same model minus the A
> >     predictor, R yields the following:
> >
> >
> >     > anova(object,objectminusA)
> >
> >     Data: data
> >     Models:
> >     objectminusA: Correct ~ 1 + (1 | Part) + (1 | Item)
> >     object: Correct ~ 1 + A + (1 | Part) + (1 | Item)
> >                  Df    AIC    BIC  logLik deviance  Chisq Chi Df
> >     Pr(>Chisq)
> >     objectminusA  3 221.64 235.53 -107.82 215.64
> >     object  5 218.91 242.04 -104.45   208.91 6.7367      2  0.03445 *
> >
> >     My questions are as follows:
> >     1. Should this be interpreted as there being a main effect but no
> >     significant difference exists between the three levels of the
> >     predictor?
> >     2. How do I report the result in my paper?