[R-sig-ME] chisq = 0 and p = 1 in glmer model comparison result

[Paul Johnson]

Hi Becky,

The model with the two additional parameters has a lower log likelihood (-480.3, df=15) than the smaller model (-475.8, df=13). This shouldn’t be possible, because adding an unconstrained parameter makes the model more flexible, allowing a closer fit to the data. This anomalous difference in likelihoods has caused the chi-squared statistic in the LRT to be negative, giving a p-value of 1, because all of the chi-squared distribution is positive, i.e. P(chi-squared > test statistic) = 1 when the test statistic is <= 0. Either one or both of the models haven’t converged, or (less likely) there is missing data in the added variable so that the data differs between the two models.

In addition, it’s very unusual to fit random slopes without also including a fixed effect, as you’ve done for targetWordFactor. The null model allows the WW-W and NW-W differences in the log odds of being correct to differ randomly between subjects and between items, but forces the mean differences (across subjects and items) to be zero. You’d need a good reason to fit such a model.

Best wishes,
Paul


> On 3 Jun 2019, at 14:25, Guillaume Adeux <guillaumesimon.a2 using gmail.com> wrote:
> 
> Hello Becky,
> 
> Even though I cannot directly answer your question... a Chisq of 0 with
> such a difference in AIC seems indeed suspicious.
> 
> To test the effect of your predictor in a GLMM context through LRT tests,
> you could (should?) consider using the test_terms function from the {monet}
> package or its little brother function mixed() from the {afex} package.
> These packages are specifically designed for such tests.
> 
> I hope this helps.
> 
> Sincerely,
> 
> Guillaume ADEUX
> 
> 
> 
> Le lun. 3 juin 2019 à 15:14, Becky Gilbert <beckyannegilbert using gmail.com> a
> écrit :
> 
>> Dear list
>> 
>> I have two glmer models, one with a fixed factor (targetWordFactor) and one
>> without, and I am comparing them using the anova function to get the LRT
>> results for the fixed effect of targetWordFactor. The anova results are
>> showing a chi-square value of 0 and p value of 1. Is this result possible,
>> or is it perhaps a sign that I've done something wrong?
>> 
>> Here are the anova results:
>> 
>> anova(accModelNullWord,accModelWord)
>> #                                  Df    AIC    BIC logLik deviance Chisq
>> Chi Df Pr(>Chisq)
>> # accModelNullWord 13 977.59 1067.0 -475.8   951.59
>> # accModelWord       15 990.61 1093.8 -480.3   960.61     0      2
>> 1
>> 
>> The targetWordFactor fixed factor has 3 levels (2 contrasts), so the
>> degrees of freedom in the anova result look correct to me. Here are the
>> model specifications:
>> 
>> contrasts(pauseDetValidNoFillersExcluded$targetWordFactor)
>> #        WW NW
>> # W       0  0
>> # WW   1  0
>> # NW    0  1
>> 
>> accModelNullWord <- glmer(correct ~ 1 +
>>                            (1 + targetWordFactor|subject) +
>>                            (1 + targetWordFactor|item),
>>                            data = pauseDetValidNoFillersExcluded,
>>                            family = binomial(link = "logit"),
>>                            control = glmerControl(optimizer="bobyqa",
>>                                                 optCtrl =
>> list(maxfun=2e5)))
>> 
>> accModelWord <- glmer(correct ~ 1 + targetWordFactor +
>>                        (1 + targetWordFactor|subject) +
>>                        (1 + targetWordFactor|item),
>>                        data = pauseDetValidNoFillersExcluded,
>>                        family = binomial(link = "logit"),
>>                        control = glmerControl(optimizer="bobyqa",
>>                                             optCtrl = list(maxfun=2e5)))
>> 
>> Apologies if this question has been asked before - I did search the list
>> but couldn't find anything.
>> 
>> Many thanks,
>> Becky
>> 
>>        [[alternative HTML version deleted]]
>> 
>> _______________________________________________
>> R-sig-mixed-models using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models