[R-sig-ME] general GLMM questions

[Ken Beath]

On 08/05/2008, at 1:39 AM, Ben Bolker wrote:

>
> 1. Is there a published justification somewhere for Lynn Eberly's
> ( http://www.biostat.umn.edu/~lynn/ph7430/class.html ) statement  
> that df
> adjustments are largely irrelevant if number of blocks>25 ?
>

Actually denominator df > 25. This seems to derive from t  
distributions with df greater than 25 all being much the same, in fact  
close to a normal distribution. In reality variations in the data from  
normality are more important.


> 2. What determines the asymptotic performance of the LRT (likelihood
> ratio test) for comparison of fixed effects, which is known to be poor
> for "small" sample sizes?  Is it the number of random-effects levels
> (as stated by Agresti 2002, p. 520), or is it the number of levels of
> the fixed effect relative to the total number of data points (as
> stated by Pinheiro and Bates 2000, pp. 87-89)?  (The example given by
> PB2000 from Littell et al. 1996 is a test of a treatment factor with
> 15 levels, in a design with 60 observations and 15 blocks.  Agresti's
> statement would imply that one would still be in trouble if the total
> number of observations increased to 600 [because # blocks is still
> small], where PB2000 would imply that the LRT would be OK in this
> limit.  (A small experiment with the simulate.lme() example given on
> PB2000 p. 89 suggests that increasing the sample size 10-fold with the
> same number of blocks DOES make the LRT OK ... but I would need to do
> this a bit more carefully to be sure.)  (Or is this a difference
> between the linear and generalized linear case?)
>

This is probably dependent on whether the comparisons are within- or  
between- block. The PBIB has lots of within- block comparisons so  
increasing block size will tend to make things asymptotic. Try blocks  
where all within a block receive the same treatment and see how much  
increasing block size helps.


> 3. For multi-level models (nested, certainly crossed), how would one
> count the "number of random-effects levels" to quantify the 'sample
> size' above?  With a single random effect, we can just count the
> number of levels (blocks).  What would one do with e.g. a nested or
> crossed design?  (Perhaps the answer is "don't use a likelihood ratio
> test to evaluate the significance of fixed effects".)
>
> 4. Does anyone know of any evidence (in either direction) that the
> "boundary" problems that apply to the likelihood ratio test (e.g. Self
> and Liang 1987) also apply to information criteria comparisons of
> models with and without random effects?  I would expect so, since the
> derivations of the AIC involve Taylor expansions around the
> null-hypothesis parameters ...
>

This is a good question. For choosing the number of classes for  
mixture models it has been shown that BIC fails theoretically but  
works well in practice (proven with simulations) and compares well to  
results from parametric bootstrapping of the LRT. Some simulations for  
random effects would be interesting.

Ken