[R-sig-ME] Fwd: glm.nb convergence issues

[Thierry Onkelinx]

Dear Matthew,

The fixed effect of your model suggests following EDA plot: ggplot(ST,
aes(x = spr, y = wait_n / wait_d)) + geom_point()

Based on this plot, I doubt if the negative binomial distribution makes
sense for your data. The problem is IMHO rather conceptual (what model
makes sense) than technical (false convergence). I would strongly recommend
to consult a local statistician.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkelinx using inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be

///////////////////////////////////////////////////////////////////////////////////////////
To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
///////////////////////////////////////////////////////////////////////////////////////////

<https://www.inbo.be>


Op wo 20 mrt. 2019 om 20:50 schreef Matthew Boden <matthew.t.boden using gmail.com
>:

> Thank you, Thierry.  A csv file is attached; hopefully, that works.
>
> The numerator divided by the denominator will always fall between 0 and 1
> for these data, and is typically very close to 1.
> The predictor variable ranges from about 3 to 18, but even when scaled,
> the model won't converge.
>
> Matt
>
>
>
>
>
>
>
>
>
>
>
>
> On Wed, Mar 20, 2019 at 1:48 AM Thierry Onkelinx <thierry.onkelinx using inbo.be>
> wrote:
>
>> Dear Matthew,
>>
>> The mailing list accepts only a limited number of file formats as
>> attachment. Your data got stripped. Can you resend the data or send a link
>> to the data?
>>
>> The offset requires the log because the model is using the log link. Your
>> model fits log(E(wait_n)) = offset(log(wait_d)) + covariates which you can
>> rewrite as log(E(wait_n)) - offset(log(wait_d)) = covariates or
>> log(E(wait_n) / wait_d) = covariates. Pick a relevant magnitude of wait_d
>> so that the magnitude of wait_n/wait_d is somewhat near 1. E.g. we don't
>> express the number of inhabitats per m² but rather per km²
>>
>> Best regards,
>>
>> ir. Thierry Onkelinx
>> Statisticus / Statistician
>>
>> Vlaamse Overheid / Government of Flanders
>> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
>> AND FOREST
>> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
>> thierry.onkelinx using inbo.be
>> Havenlaan 88 bus 73, 1000 Brussel
>> www.inbo.be
>>
>>
>> ///////////////////////////////////////////////////////////////////////////////////////////
>> To call in the statistician after the experiment is done may be no more
>> than asking him to perform a post-mortem examination: he may be able to say
>> what the experiment died of. ~ Sir Ronald Aylmer Fisher
>> The plural of anecdote is not data. ~ Roger Brinner
>> The combination of some data and an aching desire for an answer does not
>> ensure that a reasonable answer can be extracted from a given body of data.
>> ~ John Tukey
>>
>> ///////////////////////////////////////////////////////////////////////////////////////////
>>
>> <https://www.inbo.be>
>>
>>
>> Op wo 20 mrt. 2019 om 06:43 schreef Matthew Boden <
>> matthew.t.boden using gmail.com>:
>>
>>> Hello,
>>>
>>>
>>>
>>> I’m working my way through convergence issues related to a negative
>>> binomial mixed model with an offset and random intercept. Data attached.
>>>
>>>
>>>
>>> A3 <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 | check),
>>> data = ST)
>>>
>>> summary(A3)
>>>
>>>
>>>
>>> #Model failed to converge with max|grad| = 0.0031459 (tol = 0.001,
>>> component 1)
>>>
>>> #Model is nearly unidentifiable: very large eigenvalue
>>>
>>> # - Rescale variables?
>>>
>>>
>>>
>>> I have two questions.
>>>
>>>
>>>
>>> 1) Outcome variable (wait_n) values are much larger than predictor (spr)
>>> values.
>>>
>>>
>>>
>>> Mean wait_n = 11,783
>>>
>>> Mean spr = 7.34
>>>
>>>
>>>
>>> Would transforming the predictor (e.g., multiply by 1000) and/or offset
>>> make sense here. The offset confuses the issue (or, me) - as a log, it is
>>> quite small relative to the outcome and on but on the same scale as the
>>> other predictor.
>>>
>>>
>>>
>>> 2) The use of allFit to try different optimizers seems to be giving
>>> conflicting results.
>>>
>>>
>>>
>>> A3.all <- allFit(A3, meth.tab = NULL)
>>>
>>> ss <- summary(A3.all)
>>>
>>>
>>>
>>> #bobyqa : [OK]
>>>
>>> #Nelder_Mead : [OK]
>>>
>>> #nlminbwrap : [OK]
>>>
>>> #nmkbw : [OK]
>>>
>>> #optimx.L-BFGS-B : [OK]
>>>
>>> #nloptwrap.NLOPT_LN_NELDERMEAD : [OK]
>>>
>>> #nloptwrap.NLOPT_LN_BOBYQA : [OK]
>>>
>>>
>>>
>>> ss$ which.OK
>>>
>>>
>>>
>>> #bobyqa                         TRUE
>>>
>>> #Nelder_Mead                    TRUE
>>>
>>> #nlminbwrap                     TRUE
>>>
>>> #nmkbw                          TRUE
>>>
>>> #optimx.L-BFGS-B                TRUE
>>>
>>> #nloptwrap.NLOPT_LN_NELDERMEAD  TRUE
>>>
>>> #nloptwrap.NLOPT_LN_BOBYQA      TRUE
>>>
>>>
>>>
>>> What do you know, all optimizers supposedly work.
>>>
>>> However, some of them lead to a model that converges, and others don’t.
>>>
>>>
>>>
>>> A3a <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 | check),
>>> data = ST, glmerControl(optimizer = "bobyqa"))
>>>
>>>
>>>
>>> #Model is nearly unidentifiable: very large eigenvalue
>>>
>>> # - Rescale variables?
>>>
>>>
>>>
>>> A3b <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 | check),
>>> data = ST, glmerControl(optimizer = "Nelder_Mead"))
>>>
>>>
>>>
>>> #Model failed to converge with max|grad| = 0.00553513 (tol = 0.001,
>>> component 1)
>>>
>>> #Model is nearly unidentifiable: very large eigenvalue
>>>
>>> # - Rescale variables?
>>>
>>>
>>>
>>> A3c <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 | check),
>>> data = ST, glmerControl(optimizer = "nlminbwrap"))
>>>
>>>
>>>
>>> #Model is nearly unidentifiable: very large eigenvalue
>>>
>>> # - Rescale variables?
>>>
>>>
>>>
>>> A3d <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 | check),
>>> data = ST, glmerControl(optimizer = "nmkbw"))
>>>
>>>
>>>
>>> #Model failed to converge with max|grad| = 0.00177926 (tol = 0.001,
>>> component 1)
>>>
>>> #Model is nearly unidentifiable: very large eigenvalue
>>>
>>> # - Rescale variables?
>>>
>>>
>>>
>>> I’m trying to understand why allFit responds that all optimizers converge
>>> when they actually do not.
>>>
>>>
>>>
>>> Thank you,
>>>
>>> Matt
>>>
>>>
>>>
>>> Matthew Boden, Ph.D.
>>>
>>> Senior Evaluator
>>>
>>> Program Evaluation & Resource Center
>>>
>>> Office of Mental Health & Suicide Prevention
>>>
>>> Veterans Health Administration
>>> _______________________________________________
>>> R-sig-mixed-models using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>
>>

	[[alternative HTML version deleted]]