[R] Using anova(f1, f2) to compare lmer models yields seemingly erroneous Chisq = 0, p = 1

[rapton]

Thank you all for your insight!  I am glad to hear, at least, that I am doing
something incorrectly (since the results do not make sense), and I am very
grateful for your attempts to remedy my very limited (and admittedly
self-taught) understanding of multilevel models and R.

As I mentioned in the problem statement, predictor.1 explains vastly more
variance in outcome than predictor.2 (R2 = 15% vs. 5% in OLS regression,
with very large N), and the model estimates are very similar for the
multilevel model as for OLS regression.  Therefore, I am quite confident
that predictor.1 comprises a much better model.

I understand that several of you are saying that anova() cannot be used to
compare these two multilevel models.  Is there *any* way to compare two
predictors to see which better predicts the outcome in a multilevel model? 
f1's lower AIC and BIC, and higher logLik are concordant with the idea that
predictor.1 is superior to predictor.2, as best as I understand it, but is
there any way to test whether that difference is statistically significant? 
The only function I can find online is anova() to compare models, but its
output is nonsensical and, as you are all saying, it does not apply to my
situation anyway.

Interestingly, anova() seems to "work" if I arbitrarily subset my
observations, but when I use all the observations anova() generates Chisq =
0.  This is probably a red herring but I thought I would mention it in case
it is not.

Also, I concede that I am confused what you mean that the two models (f1 and
f2) are not nested, and therefore anova() cannot be used.  What would be an
example of a nested model:  comparing predictor.1 to a model with both
predictor.1 and predictor.2?  Surely there must also be a way to compare the
predictive power of predictor.1 and predictor.2 to each other in a
"zero-order" sense, but I am at a loss to identify it.



Alain Zuur wrote:
> 
> 
> 
> rapton wrote:
>> 
>>  When I run two models, the output of each model is generated
>> correctly as far as I can tell (e.g. summary(f1) and summary(f2) for the
>> multilevel model output look perfectly reasonable), and in this case (see
>> below) predictor.1 explains vastly more variance in outcome than
>> predictor.2
>> (R2 = 15% vs. 5% in OLS regression, with very large N).  What I am
>> utterly
>> puzzled by is that when I run an anova comparing the two multilevel model
>> fits, the Chisq comes back as 0, with p = 1.  I am pretty sure that fit
>> #1
>> (f1) is a much better predictor of the outcome than f2, which is
>> reflected
>> in the AIC, BIC , and logLik values.  Why might anova be giving me this
>> curious output?  How can I fix it?  I am sure I am making a dumb error
>> somewhere, but I cannot figure out what it is.  Any help or suggestions
>> would 
>> be greatly appreciated!
>> 
>> -Matt
>> 
>> 
>>> f1 <- (lmer(outcome ~ predictor.1 + (1 | person), data=i))
>>> f2 <- (lmer(outcome ~ predictor.2 + (1 | person), data=i))
>>> anova(f1, f2)
>> 
>> Data: i
>> Models:
>> f1: outcome ~ predictor.1 + (1 | person)
>> f2: outcome ~ predictor.2 + (1 | person)
>>    Df    AIC      BIC    logLik   Chisq Chi Df Pr(>Chisq)
>> f1  6  45443  45489 -22715
>> f2 25  47317  47511 -23633     0     19          1
>> 
> 
> 
> 
> 
> Your models are nest nested....it doesn't make sense to do. 
> 
> 
> Alain
> 

-- 
View this message in context: http://www.nabble.com/Using-anova%28f1%2C-f2%29-to-compare-lmer-models-yields-seemingly-erroneous-Chisq-%3D-0%2C-p-%3D-1-tp25297254p25338046.html
Sent from the R help mailing list archive at Nabble.com.