[R-sig-ME] lme4/glmer convergence warnings

[Ben Bolker]

On 14-04-06 04:31 AM, Tibor Kiss wrote:
> Hi,
> 
> being somewhat nonplussed by similar messages, I also applied Ben's recent suggestion to one of my models to get:
> 
>      Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
> 1.343e-05 3.530e-05 5.756e-05 7.631e-05 9.841e-05 1.932e-04 
> 
> So following up on Rob's message: What does it mean?
> 
> With kind regards
> 
> Tibor
> 

  It means that on the scale of the _standard deviations_ of the
parameters, the estimated gradients at the MLE (or restricted MLE) are
not large.  I was surprised in Rob's case that these scaled gradients
were not that small; much smaller than without the scaling, but not
small enough to make me think  really understand what's going on.

  To recapitulate: the appearance of all of these new messages in the
latest version of lme4 is **not** due to a degradation or change in the
optimization or fitting procedure -- it's due to a new set of
convergence tests that we implemented, that we think are giving a lot of
false positives.  You can easily shut them off yourself, or raise the
tolerance for the warnings (see ?lmerControl/?glmerControl).  As
developers, we're a bit stuck now because we don't want to turn the
warnings _off_ until we understand the circumstances that are triggering
them better, and that takes more time and effort than we've been able to
muster so far.

   cheers
     Ben Bolker


> 
> 
>  
> 
> Am 04.04.2014 um 16:50 schrieb W Robert Long <longrob604 at gmail.com>:
> 
>> Hi Ben
>>
>> Does the output I posted in my earlier email help ?
>>
>> Thanks
>> Rob
>>
>> On 02/04/2014 20:25, W Robert Long wrote:
>>> Hi Ben
>>>
>>> Thanks for your reply. The code you posted generates the following:
>>>
>>>  Min.   1st Qu.    Median     Mean   3rd Qu.     Max.
>>> 0.001474 0.023920 0.045420 0.255600 0.068600 2.114000
>>>
>>> This model was fitted with the raw data (not standardised continuous
>>> data) and without removing small clusters.
>>>
>>> Thanks again
>>> Robert Long
>>>
>>>
>>>
>>>
>>> On 02/04/2014 14:05, Ben Bolker wrote:
>>>>
>>>>   I think this is a false positive, caused by our recent introduction of
>>>> new convergence tests. There's been lots of discussion of this on the
>>>> list recently.
>>>>
>>>>    I have a new trouble-shooting idea:
>>>>
>>>> if g0 is your fitted model, can you see what happens if you scale the
>>>> estimated gradients by the curvature/standard errors?
>>>>
>>>> gg <- g0 at optinfo$derivs$grad
>>>> hh <- g0 at optinfo$derivs$Hessian
>>>> vv <- sqrt(diag(solve(hh/2)))
>>>> summary(abs(gg*vv))
>>>>
>>>>
>>>> On 14-04-02 06:40 AM, W Robert Long wrote:
>>>>> I should perhaps also mention that of the 9 covariates, 3 are continous
>>>>> and I have tried standardising them. Of the remaining 6, 5 are binary
>>>>> and the last one is ordinal.
>>>>>
>>>>> On 02/04/2014 11:28, W Robert Long wrote:
>>>>>> Hi all
>>>>>>
>>>>>> I am running a simple random intercepts model using lme4 on
>>>>>> approximately 70,000 observations, with 250 clusters. The code looks
>>>>>> like
>>>>>>
>>>>>> glmer(Y~x1+x2+x3+x4+x5+x6+x7+x8+x9+(1|clusdID),
>>>>>>      data=dt1, family=binomial(link=logit))
>>>>>>
>>>>>> and I receive the following warnings:
>>>>>>
>>>>>> Warning messages:
>>>>>> 1: In checkConv(attr(opt, "derivs"), opt$par, ctrl =
>>>>>> control$checkConv,  :
>>>>>>    Model failed to converge with max|grad| = 4847.75 (tol = 0.001)
>>>>>> 2: In if (resHess$code != 0) { :
>>>>>>    the condition has length > 1 and only the first element will be
>>>>>> used
>>>>>> 3: In checkConv(attr(opt, "derivs"), opt$par, ctrl =
>>>>>> control$checkConv,  :
>>>>>>    Model is nearly unidentifiable: very large eigenvalue
>>>>>>   - Rescale variables?;Model is nearly unidentifiable: large
>>>>>> eigenvalue
>>>>>> ratio
>>>>>>   - Rescale variables?
>>>>>>
>>>>>> There are some small clusters (<10 obs per cluster), but even removing
>>>>>> those, the warnings remain.
>>>>>>
>>>>>> Using Stata -xtmelogit- there are no warnings and the output is almost
>>>>>> identical to glmer() so this gives me some comfort, yet I still worry
>>>>>> about these warnings from glmer.
>>>>>>
>>>>>> I have tried setting nAGQ as high as 10, to no avail.
>>>>>>
>>>>>> Could anyone suggest what I can look for or change ? The data are
>>>>>> confidential so I can't easily make a reprodicible example.
>>>>>>
>>>>>> Thanks in advance
>>>>>> Robert Long
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> R-sig-mixed-models at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>