[R-sig-ME] Fwd: glm.nb convergence issues

[Matthew Boden]

Thank you, Thierry. Will do.

Matt

On Fri, Mar 22, 2019 at 7:29 AM Thierry Onkelinx <thierry.onkelinx using inbo.be>
wrote:

> Dear Matthew,
>
> The fixed effect of your model suggests following EDA plot: ggplot(ST,
> aes(x = spr, y = wait_n / wait_d)) + geom_point()
>
> Based on this plot, I doubt if the negative binomial distribution makes
> sense for your data. The problem is IMHO rather conceptual (what model
> makes sense) than technical (false convergence). I would strongly recommend
> to consult a local statistician.
>
> Best regards,
>
> ir. Thierry Onkelinx
> Statisticus / Statistician
>
> Vlaamse Overheid / Government of Flanders
> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
> FOREST
> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
> thierry.onkelinx using inbo.be
> Havenlaan 88 bus 73, 1000 Brussel
> www.inbo.be
>
>
> ///////////////////////////////////////////////////////////////////////////////////////////
> To call in the statistician after the experiment is done may be no more
> than asking him to perform a post-mortem examination: he may be able to say
> what the experiment died of. ~ Sir Ronald Aylmer Fisher
> The plural of anecdote is not data. ~ Roger Brinner
> The combination of some data and an aching desire for an answer does not
> ensure that a reasonable answer can be extracted from a given body of data.
> ~ John Tukey
>
> ///////////////////////////////////////////////////////////////////////////////////////////
>
> <https://www.inbo.be>
>
>
> Op wo 20 mrt. 2019 om 20:50 schreef Matthew Boden <
> matthew.t.boden using gmail.com>:
>
>> Thank you, Thierry.  A csv file is attached; hopefully, that works.
>>
>> The numerator divided by the denominator will always fall between 0 and 1
>> for these data, and is typically very close to 1.
>> The predictor variable ranges from about 3 to 18, but even when scaled,
>> the model won't converge.
>>
>> Matt
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Mar 20, 2019 at 1:48 AM Thierry Onkelinx <
>> thierry.onkelinx using inbo.be> wrote:
>>
>>> Dear Matthew,
>>>
>>> The mailing list accepts only a limited number of file formats as
>>> attachment. Your data got stripped. Can you resend the data or send a link
>>> to the data?
>>>
>>> The offset requires the log because the model is using the log link.
>>> Your model fits log(E(wait_n)) = offset(log(wait_d)) + covariates which you
>>> can rewrite as log(E(wait_n)) - offset(log(wait_d)) = covariates or
>>> log(E(wait_n) / wait_d) = covariates. Pick a relevant magnitude of wait_d
>>> so that the magnitude of wait_n/wait_d is somewhat near 1. E.g. we don't
>>> express the number of inhabitats per m² but rather per km²
>>>
>>> Best regards,
>>>
>>> ir. Thierry Onkelinx
>>> Statisticus / Statistician
>>>
>>> Vlaamse Overheid / Government of Flanders
>>> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
>>> AND FOREST
>>> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
>>> thierry.onkelinx using inbo.be
>>> Havenlaan 88 bus 73, 1000 Brussel
>>> www.inbo.be
>>>
>>>
>>> ///////////////////////////////////////////////////////////////////////////////////////////
>>> To call in the statistician after the experiment is done may be no more
>>> than asking him to perform a post-mortem examination: he may be able to say
>>> what the experiment died of. ~ Sir Ronald Aylmer Fisher
>>> The plural of anecdote is not data. ~ Roger Brinner
>>> The combination of some data and an aching desire for an answer does not
>>> ensure that a reasonable answer can be extracted from a given body of data.
>>> ~ John Tukey
>>>
>>> ///////////////////////////////////////////////////////////////////////////////////////////
>>>
>>> <https://www.inbo.be>
>>>
>>>
>>> Op wo 20 mrt. 2019 om 06:43 schreef Matthew Boden <
>>> matthew.t.boden using gmail.com>:
>>>
>>>> Hello,
>>>>
>>>>
>>>>
>>>> I’m working my way through convergence issues related to a negative
>>>> binomial mixed model with an offset and random intercept. Data attached.
>>>>
>>>>
>>>>
>>>> A3 <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 | check),
>>>> data = ST)
>>>>
>>>> summary(A3)
>>>>
>>>>
>>>>
>>>> #Model failed to converge with max|grad| = 0.0031459 (tol = 0.001,
>>>> component 1)
>>>>
>>>> #Model is nearly unidentifiable: very large eigenvalue
>>>>
>>>> # - Rescale variables?
>>>>
>>>>
>>>>
>>>> I have two questions.
>>>>
>>>>
>>>>
>>>> 1) Outcome variable (wait_n) values are much larger than predictor (spr)
>>>> values.
>>>>
>>>>
>>>>
>>>> Mean wait_n = 11,783
>>>>
>>>> Mean spr = 7.34
>>>>
>>>>
>>>>
>>>> Would transforming the predictor (e.g., multiply by 1000) and/or offset
>>>> make sense here. The offset confuses the issue (or, me) - as a log, it
>>>> is
>>>> quite small relative to the outcome and on but on the same scale as the
>>>> other predictor.
>>>>
>>>>
>>>>
>>>> 2) The use of allFit to try different optimizers seems to be giving
>>>> conflicting results.
>>>>
>>>>
>>>>
>>>> A3.all <- allFit(A3, meth.tab = NULL)
>>>>
>>>> ss <- summary(A3.all)
>>>>
>>>>
>>>>
>>>> #bobyqa : [OK]
>>>>
>>>> #Nelder_Mead : [OK]
>>>>
>>>> #nlminbwrap : [OK]
>>>>
>>>> #nmkbw : [OK]
>>>>
>>>> #optimx.L-BFGS-B : [OK]
>>>>
>>>> #nloptwrap.NLOPT_LN_NELDERMEAD : [OK]
>>>>
>>>> #nloptwrap.NLOPT_LN_BOBYQA : [OK]
>>>>
>>>>
>>>>
>>>> ss$ which.OK
>>>>
>>>>
>>>>
>>>> #bobyqa                         TRUE
>>>>
>>>> #Nelder_Mead                    TRUE
>>>>
>>>> #nlminbwrap                     TRUE
>>>>
>>>> #nmkbw                          TRUE
>>>>
>>>> #optimx.L-BFGS-B                TRUE
>>>>
>>>> #nloptwrap.NLOPT_LN_NELDERMEAD  TRUE
>>>>
>>>> #nloptwrap.NLOPT_LN_BOBYQA      TRUE
>>>>
>>>>
>>>>
>>>> What do you know, all optimizers supposedly work.
>>>>
>>>> However, some of them lead to a model that converges, and others don’t.
>>>>
>>>>
>>>>
>>>> A3a <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 |
>>>> check),
>>>> data = ST, glmerControl(optimizer = "bobyqa"))
>>>>
>>>>
>>>>
>>>> #Model is nearly unidentifiable: very large eigenvalue
>>>>
>>>> # - Rescale variables?
>>>>
>>>>
>>>>
>>>> A3b <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 |
>>>> check),
>>>> data = ST, glmerControl(optimizer = "Nelder_Mead"))
>>>>
>>>>
>>>>
>>>> #Model failed to converge with max|grad| = 0.00553513 (tol = 0.001,
>>>> component 1)
>>>>
>>>> #Model is nearly unidentifiable: very large eigenvalue
>>>>
>>>> # - Rescale variables?
>>>>
>>>>
>>>>
>>>> A3c <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 |
>>>> check),
>>>> data = ST, glmerControl(optimizer = "nlminbwrap"))
>>>>
>>>>
>>>>
>>>> #Model is nearly unidentifiable: very large eigenvalue
>>>>
>>>> # - Rescale variables?
>>>>
>>>>
>>>>
>>>> A3d <- glmer.nb(wait_n ~ offset(log(wait_d)) +  scale(spr) + (1 |
>>>> check),
>>>> data = ST, glmerControl(optimizer = "nmkbw"))
>>>>
>>>>
>>>>
>>>> #Model failed to converge with max|grad| = 0.00177926 (tol = 0.001,
>>>> component 1)
>>>>
>>>> #Model is nearly unidentifiable: very large eigenvalue
>>>>
>>>> # - Rescale variables?
>>>>
>>>>
>>>>
>>>> I’m trying to understand why allFit responds that all optimizers
>>>> converge
>>>> when they actually do not.
>>>>
>>>>
>>>>
>>>> Thank you,
>>>>
>>>> Matt
>>>>
>>>>
>>>>
>>>> Matthew Boden, Ph.D.
>>>>
>>>> Senior Evaluator
>>>>
>>>> Program Evaluation & Resource Center
>>>>
>>>> Office of Mental Health & Suicide Prevention
>>>>
>>>> Veterans Health Administration
>>>> _______________________________________________
>>>> R-sig-mixed-models using r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>>>

	[[alternative HTML version deleted]]