[R-sig-ME] Chisq = 0 and p = 1 with nested lmer models

[Geoff Brookshire]

Dear all,

I'm using lme4 to analyze some reaction time data I've collected, and
am running into an odd problem. I'm not certain if it's a bug or a
problem with my thinking, but hopefully (and probably) it's the latter.

When trying to compare two models, one of which is the same as the
other but with one term removed, the result is that there is no
difference between the models: Chisq = 0, Chisq Df = 0, and p = 1.

A little about the design:
This is a reaction-time experiment with a 2x2x2 design over factors
F1, F2, and F3. F1 is varied within-subjects and within-items, and F2
& F3 are varied within-subjects and between-items.

So far, I've been testing the models like this:

fm.1 <- lmer(outcome ~ F1*F2*F3 + (1 | subject) + (1 | item),
   data = samp.dat, REML = FALSE)
fm.2 <- lmer(outcome ~ F1*F2*F3 - F1:F2:F3 + (1 | subject) + (1 | item),
   data = samp.dat, REML = FALSE)
anova(fm.1, fm.2)

This works, and gives what we expect. When I try to test the 2-term
interactions, though, the error comes up:

fm.3 <- lmer(outcome ~ F1*F2*F3 - F1:F2 + (1 | subject) + (1 | item),
   data = samp.dat, REML = FALSE)
anova(fm.1, fm.3)

This gives the following output:

fm.1: outcome ~ F1 * F2 * F3 + (1 | subject) + (1 | item)
fm.3: outcome ~ F1 * F2 * F3 - F1:F2 + (1 | subject) + (1 | item)
     Df     AIC     BIC logLik Chisq Chi Df Pr(>Chisq)
fm.1  11 -677.34 -617.72 349.67
fm.3  11 -677.34 -617.72 349.67     0      0          1

The same thing happens when looking at F2:F3 and F1:F3. Any ideas?
Just entering anova(fm.1) gives sensible results and F-values for the
2-way interactions.

I've seen one similar question online
(http://www.mail-archive.com/r-help@r-project.org/msg68216.html), but
he was comparing models that weren't nested -- not the problem here.

Thanks, R crew!

Geoff Brookshire