[R] zero random effect sizes with binomial lmer

[Ken Beath]

On Sun, 31 Dec 2006, at 05:50 PM, Daniel Ezra Johnson  
<johnson4 at babel.ling.upenn.edu> wrote:

>
> Gregor,
>
> Thanks for your replies.
>
> 1) Yes, I have tweaked the data to show as clearly as I can that  
> this is a
> bug, that a tiny change in initial conditions causes the collapse of a
> reasonable 'parameter' estimate.
>
> 2) mcmcsamp() does not work (currently) for binomial fitted models.
>
> 3) This is an issue of what happens when the sample is too small.  
> For all
> larger data sets I have gotten a ranef variance between 0.05 and  
> 1.00 or
> so.
>
> It makes no sense to say that as the data set gets smaller, the  
> systematic
> variation between Items goes away. It doesn't, as I've shown. In  
> the data
> above, certain Items were still 10+ times as likely (log-odds wise) to
> have Response==1 as others.
>
> It may make sense to say that the effect becomes unestimable, due  
> to its
> small size. But my understanding is not that this should make the
> algorithm return zero as an estimated value.
>
> D
>
>

There is always the possibility that the Laplace approximation is  
proving too inaccurate for this problem but that seems unlikely, as  
there should be enough observations. The only way to check is to use  
a package that uses adaptive Gauss-Hermite for the integration,  
gllamm in Stata or NLMIXED in SAS may do it. PQL is even worse, so it  
is not an option.

The real problem is that there is not enough variation across your  
items,  and so the estimate of the random effect is close to zero.  
The only difference between your datasets is that one results in an  
estimate closer to zero. Fit the model without (Item|1), and the fit  
hardly worsens in either case, with resulting better AIC and BIC.

The responses by item can be checked with
 > apply(aa,2,sum)
[1] 2 2 2 4 1 5 4
 > apply(bb,2,sum)
[1] 2 2 2 4 1 6 4

This is not perfect as it ignores the subject variation, but they  
don't have a lot of variation.