[R-sig-ME] glmm AIC/LogLik reliability

[Virgilio Gomez Rubio]

Hi,

>    I think AIC is no worse than anything else in this regard,
> except that it hasn't been explored as carefully
> as some of the alternatives: thus we suspect by analogy
> that there are problems similar to those of the LRT,
> but we don't know for sure.
> Vaida and Blanchard (2005), Greven (2008), and Burnham
> and White (2002) are good references.  There are

I would also point to the paper by Spiegelhalter et al. (2002) on the
DIC. It is a 'Bayesian version' of the DIC but the examples and
discussions therein are quite interesting.

> two basic issues:
>   (1) if you choose to include models that differ
> in their random effects components, how do you count
> "effective" degrees of freedom?
>   (2) how big a sample does it take to reach the
> "asymptopia" of AIC?  If you're not there, what is
> the best strategy for finite-size correction?  If
> you use AICc, what should you put in for effective
> residual degrees of freedom?

We are trying to make a comparison of AIC, cAIC (Vaida and Blanchard,
2005) and DIC in this working paper:

http://www.bias-project.org.uk/papers/ComparisonSAE.pdf

I believe it is a bit of an unfinished work but we have computed several
linear (mixed) models in the context of Small Area Estimation and we
display the values of AIC/cAIC/DIC in a table for comparison purposes
together with the penalty terms. The aim is to study up to what point
the AIC, cAIC and DIC are comparable using different structures for the
random effects. Any comments are welcome.

Hope this helps.

Virgilio

P.S: Is there any way of obtaining the design matrix of the random
effects and the matrix of the variance from an lme object. That would
help to compute the cAIC more easily.



> 
>    Ben Bolker
> 
> 
> D O S Gillespie wrote:
> > Dear R-Sig-ME -
> > 
> > Lets assume that I am going to use a model averaging AIC based  
> > approach to evaluate nested glmm's.
> > 
> > I would like to assume that the estimation of AIC and LogLik in the  
> > glmm's of lmer are consistent enough (precise, if not accurate) to use  
> > in this framework. I realize that we don't trust anova(m1, m2), mainly  
> > due to df and tests statistics issues.
> > 
> > I realise that some of you may suggest that this is not the correct  
> > framework.  If so, can you distinguish arguments about the philosophy  
> > of AIC model averaging from the practical implementation - i.e. is the  
> > output consistent enough to use if, even if you don't believe the  
> > answer.  Perhaps they are too intertwined.
> > 
> > Thanks,
> > 
> > Duncan Gillespie
> > 
> > _______________________________________________
> > R-sig-mixed-models at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> 
>