[R-sig-ME] likelihood-ratio tests in conflict with coefficiants in maximal random effect model

[Ben Bolker]

On 14-02-28 11:27 AM, Douglas Bates wrote:
> On Fri, Feb 28, 2014 at 10:04 AM, Emilia Ellsiepen <
> emilia.ellsiepen at gmail.com> wrote:
> 
>> Dear list members,
>>
>> in analyzing a data set using lmers with maximal random effect
>> structure and subsequent likelihood-ratio tests (LRTs) following Barr
>> et al. 2013, I ran into the following problem: In some of the LRTs, it
>> turned out that the simpler model (only main effects) has a higher
>> likelihood than the more complicated model (including interaction),
>> resulting in Chi=0. If I simplify the models by taking out the
>> interactions in the two random effect terms, the LRT for the
>> interaction has a highly significant result.>

[snip]

> It shows that the more complex model has not converged to the optimum
> parameter values.  This can be because the optimizer being used is a bad
> choice (in recent versions of the lme4 package the default was a
> Nelder-Mead optimizer that can declare convergence to values that are not
> the optimal values) or it can be because the model is too complex.  We say
> that such models are over-parameterized.
> 
> This is why the Barr et al. advice is dangerous.  In model selection there
> are two basic strategies: forward and backward.  Forward selection starts
> with a simple model and adds terms until they are no longer justified.
>  Backward selection starts with the most complex model and drops terms.  It
> is well known that backward selection is problematic when you can't fit the
> most complex model.  Yet Barr et al. say unequivocally that you must use
> backward selection.  The result will be that many researchers, especially
> in linguistics, will encounter these problems of complex models providing
> worse fits than simpler models.
> 
> I wish that Barr et al. would have provided software that is guaranteed to
> fit even the most complex model to a global optimum when they stated their
> "one size fits all" strategy.  they didn't and those with experience in
> numerical optimization can tell you why.  It is not possible to guarantee
> convergence to an optimum in over-parameterized models.
> 

  [snip]

  I would be very interested to know whether the more thorough
convergence tests that we have implemented in the development version of
lme4 would correctly report that there are convergence problems with the
model containing the maximal RE structure ...

  Ben Bolker