[R-sig-ME] Reconciling Near Identical AIC Values and Highly Significant P Value (Ben Bolker)

[Ed Merkle]

Ben et al,
>     I think you might have to back up and think about what hypothesis
> you're testing when you're comparing two non-nested models.  You could
> consider Vuong's test
> http://en.wikipedia.org/wiki/Vuong%27s_closeness_test  ;
> http://fisher.osu.edu/~schroeder.9/AMIS900/Vuong1989.pdf  ...
> alternatively, I do think comparing AICs makes sense.  AIC(model,model2)
> will just give you a list of AIC values.  bbmle::AICtab(model,model2)
> will give you a slightly prettier output.
>     Keep the various limitations of AIC (asymptotic; assumes internal
> points -- seehttp://glmm.wikidot.com/faq) in mind too.
>
>    Ben Bolker
>
>
Slightly off topic, but speaking of Vuong's tests, I wonder about the 
ease with which we could obtain the necessary output from lme4. To carry 
out the tests, I believe we need individual observations' contributions 
to both the likelihood and gradient (evaluated at the ML estimates).  I 
believe these are not simple to obtain from models fit in lme4, but I 
wonder whether you have any hints here.

Thanks,
Ed

-- 
Ed Merkle, PhD
Assistant Professor
Department of Psychological Sciences
University of Missouri
Columbia, MO, USA 65211