[R-sig-ME] glmm AIC/LogLik reliability

[Ben Bolker]

Andrew Beckerman wrote:
> Perhaps the question was not clear enough (I helped Duncan try and  
> articulate this....)
> 
> Lets assume that we maintain random effects structure in all models,  
> but we have a large multiple regression problem in the fixed effects  
> (say 8 variables potentially affecting reproduction in a  population).

  Maintaining the random effects structure takes care of the issue
of counting degrees of freedom for random effects, EXCEPT in the
finite-data (AICc or equivalent) case.
> 
> Can we assume that the LogLik calculations work in this instance?

  I would guess that you would get correct log-likelihoods/deviances
in this case, if you use ML rather than REML.  (These will essentially
be marginal deviances, integrated over the random effects.)

> If we can say yes to this, then we can assume that some calculation of  
> AIC is possible. The adjustement of the LogLik by # of paramters can  
> be manipulated by the researcher, deciding on what df means to him or  
> her, etc.  The crux of the questions is not whether inference is  
> correct, but whether the bits/mechanics about getting an AIC value for  
> a set of nested models with the same random effects are internally  
> consistent.

   If you're not worried about inference, then I'd say you're OK.
Likelihood/deviance should correctly rank models with the same degree
of complexity.  But I don't see how you're going to be able to
confidently rank models unless (a) your Ns are so large
that you can assert that you are in "asymptopia" (and N here means
both (?) number of random-effects units and total sample size)
or (b) you can figure out how to inflate penalties based on
"residual df" ...

  As always, I'm happy to be corrected.

  [blatant plug: I have a GLMM paper available online now
<http://dx.doi.org/10.1016/j.tree.2008.10.008> although much of what
it says will be well known to everyone here ...]

  Ben Bolker