[R-sig-ME] question about convergence warning and some odd-looking odds ratios in a glmer model
Emma Mellor
emm@@me||or @end|ng |rom br|@to|@@c@uk
Tue Jun 28 14:24:40 CEST 2022
Hi Thierry,
Thanks for your email, and apologies for my slow reply – I wanted to have a play about with my data a bit more before replying.
> Running m1 without specifying the optimiser made the model run.
OK, thank you. Do you know why that worked? I’d like to try and understand why that might be contributing to the problem, please.
In subsequent models with extra predictors (describing the home environment and demographics), I’ve had instances in which what I *think* is the default optimiser (Nelder_Mead) failed, yet the bobyqa ran OK. Does it sound like it’s therefore better to use bobyqa regardless?
> See if you can cluster the owners into some sensible groups or use some covariates describing the home situation of the bird.
And thanks for this too. We don’t have additional data on the owners to cluster them, I’m afraid. But we do have variables describing the home environment and demographics – I’m taking a look at these now/
Thanks again for your advice! Really appreciate it.
Best wishes,
Emma
From: Thierry Onkelinx <thierry.onkelinx using inbo.be>
Sent: 24 June 2022 14:53
To: Emma Mellor <emma.mellor using bristol.ac.uk>
Cc: r-sig-mixed-models using r-project.org
Subject: Re: [R-sig-ME] question about convergence warning and some odd-looking odds ratios in a glmer model
Dear Emma,
Running m1 without specifying the optimiser made the model run. However it returns extreme estimates for the fixed effects and the random effect variance. The problem is in the fact that most owners have only one bird. This makes the random effect nearly behave as an observation level random effect.
See if you can cluster the owners into some sensible groups or use some covariates describing the home situation of the bird.
Best regards,
ir. Thierry Onkelinx
Statisticus / Statistician
Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkelinx using inbo.be<mailto:thierry.onkelinx using inbo.be>
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be<http://www.inbo.be>
///////////////////////////////////////////////////////////////////////////////////////////
To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. ~ John Tukey
///////////////////////////////////////////////////////////////////////////////////////////
[https://inbo-website-prd-532750756126.s3-eu-west-1.amazonaws.com/inbologoleeuw_nl.png]<https://www.inbo.be/>
Op do 23 jun. 2022 om 19:10 schreef Emma Mellor <emma.mellor using bristol.ac.uk<mailto:emma.mellor using bristol.ac.uk>>:
Hi all,
I'm running glmer models in lme4 on: R version 4.1.2,
Platform: x86_64-w64-mingw32/x64 (64-bit),
Running under: Windows 10 x64 (build 19044)
My outcome (presence/absence of abnormal behaviour in pet birds) is binary, so I used family = binomial. Random effects are 'Owner_ID' - in reality, very few birds share a household, and the vast majority are single birds. I'm getting warning messages about convergence when 'Species_ID' is included the model as a predictor (I do not get error messages with other predictors - it's just this one). I've tabulated the data and I don't see any obvious reason for the error, such as missing/very few cases per levels within variables, so I'm at a bit of a loss as what to do about it. Copy of warning message next, and I'll provide a copy of my code below:
Warning messages:
1: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 0.0562293 (tol = 0.002, component 1)
2: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model is nearly unidentifiable: very large eigenvalue
- Rescale variables?
Species does have a significant effect on my outcome (which is expected), so I've spilt the dataset by species to do pairwise comparisons to try and work out what's what. When I do so, I don't get the warnings but I do get, in some cases, enormous odds ratios (and one set of normal-looking ones too).
Perhaps I'm over-reacting? But given the previous warnings I got with the initial (all four species) model, it gave me reason to pause and want to check.
Can anyone advise about what I should do about the warnings, and whether I can trust those odds ratios, please?
Thanks for any help you're able to give!
Best wishes,
Emma
Code to demonstrate here:
library(lme4)
library(dplyr)
data<-structure(list(Owner_ID = c(62L, 222L, 147L, 187L, 407L, 208L, 205L, 348L, 29L, 244L,
146L, 414L, 278L, 528L, 433L, 1039L, 930L, 902L, 1466L, 1977L,
704L, 2020L, 1423L, 782L, 308L, 291L, 61L, 1512L, 1164L, 1164L,
28L, 1549L, 1882L, 1181L, 569L, 135L, 609L, 1059L, 663L, 1465L,
1207L, 713L, 1420L, 1318L, 1357L, 1727L, 1415L, 1572L, 948L,
948L, 948L, 1897L, 1649L, 1162L, 974L, 1153L, 1802L, 1412L, 872L,
1708L, 1360L, 1616L, 1960L, 1960L, 1965L, 1794L, 636L, 339L,
1783L, 1783L, 1594L, 1558L, 1695L, 1822L, 1700L, 1667L, 869L,
534L, 1667L, 524L, 1684L, 782L, 782L, 664L, 1700L, 67L, 1416L,
1418L, 1352L, 1321L, 1569L, 598L, 821L, 604L, 2013L, 1707L, 785L,
1534L, 1299L, 565L, 1284L, 1244L, 1385L, 853L, 1014L, 1542L,
1771L, 1876L, 1063L, 1212L, 1000L, 1784L, 1439L, 547L, 745L,
717L, 702L, 1829L, 1503L, 862L, 667L, 1135L, 1541L, 1507L, 1507L,
680L, 1597L, 741L, 801L, 856L, 1295L, 1295L, 955L, 1850L, 1769L,
1393L, 1393L, 1291L, 1111L, 1405L, 728L, 1993L, 901L, 566L, 1020L,
1293L, 1320L, 1087L, 1176L, 1898L, 1940L, 1177L, 1296L, 1462L,
863L, 1918L, 1854L, 1508L, 747L, 1647L, 1671L, 1646L, 1949L,
1094L, 1184L, 1455L, 899L, 1627L, 804L, 1490L, 1768L, 555L, 735L,
2022L, 1317L, 1620L, 1605L, 919L, 597L, 1849L, 1336L, 1898L,
29L, 1392L, 1438L, 1198L, 1943L, 1139L, 1716L, 1986L, 962L, 1372L,
1673L, 1640L, 1640L, 1805L, 1163L, 1172L, 1165L, 1915L, 1915L,
1576L, 1526L, 1436L, 1998L, 1590L, 906L, 770L, 1398L, 656L, 1207L,
1120L, 805L, 805L, 1437L, 1682L, 800L, 1543L, 1238L, 2008L, 1069L,
1243L, 1326L, 1170L, 802L, 873L, 943L, 1330L, 1586L, 1833L, 1717L,
1654L, 1748L, 881L, 1006L, 1637L, 1701L, 1823L, 1613L, 1599L,
875L, 1813L, 1725L, 1005L, 1322L, 1839L, 352L, 1674L, 1358L,
1688L, 1724L, 1367L, 1827L, 1286L, 1150L, 1780L, 842L, 1118L,
884L, 1440L, 173L, 1744L, 578L, 1635L, 1568L, 861L, 1091L, 650L,
1354L, 1368L, 815L, 721L, 1787L, 1787L, 1678L, 1864L, 1786L,
1698L, 1984L, 1612L, 1159L, 1611L, 822L, 1042L, 1042L, 858L,
858L, 1731L, 1655L, 993L, 160L, 478L, 445L, 239L, 531L, 303L,
18L, 171L, 531L, 492L, 159L, 74L, 456L, 488L, 305L, 423L, 477L,
467L, 9L, 464L, 102L, 232L, 131L, 118L, 143L, 34L, 196L, 30L,
153L, 412L, 371L, 104L, 16L, 63L, 248L, 212L, 482L, 480L, 266L,
259L, 283L, 283L, 148L, 156L, 523L, 368L, 513L, 241L, 246L, 441L,
229L, 320L, 245L, 287L, 468L, 468L, 383L, 367L, 54L, 380L, 25L,
37L, 73L, 496L, 546L, 382L, 369L, 236L, 113L, 365L, 499L, 227L,
465L, 425L, 1819L, 657L, 635L, 1910L, 924L, 924L, 1485L, 1289L,
627L, 1076L, 627L, 1219L, 874L, 1289L, 1292L, 1754L, 972L, 1084L,
1240L, 777L, 609L, 626L, 1500L, 788L, 1470L, 1136L, 582L, 1891L,
645L, 1557L, 339L, 1904L, 2029L, 1107L, 665L, 631L, 631L, 1214L,
1214L, 738L, 973L, 888L, 1919L, 1772L, 1772L, 351L, 314L, 342L,
1077L, 134L, 273L, 495L, 166L, 514L, 247L, 203L, 125L, 2026L,
154L, 1570L, 1947L, 1055L, 1231L, 1318L, 1437L, 626L, 206L, 164L,
708L, 1711L, 922L, 1900L, 93L, 385L, 84L, 1173L, 294L, 470L,
23L, 1113L, 870L, 870L, 1029L, 2009L, 963L, 1746L, 988L, 1108L,
1749L, 1749L, 1749L, 1749L, 1828L, 1103L, 1844L, 970L, 769L,
1188L, 1712L, 1188L, 1776L, 1901L, 860L, 1777L, 1325L, 1608L,
1828L, 1608L, 1542L, 328L, 1942L, 1843L, 967L, 1990L, 1852L,
1056L, 1847L, 204L, 1338L, 846L, 729L, 1393L, 864L, 1396L, 1393L,
1777L),
Outcome = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L,
1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L,
0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L,
0L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 1L,
0L, 0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L,
0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L,
1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L,
1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L,
0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L,
0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 1L,
1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 1L,
0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L,
0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L,
1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 1L, 1L, 0L, 1L, 0L, 1L, 0L,
0L, 0L, 1L, 0L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L,
0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 1L,
0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L,
0L, 1L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
1L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L,
1L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 0L,
1L, 1L, 1L, 1L, 1L, 0L, 0L, 1L, 0L, 1L, 1L, 0L, 0L, 1L, 0L, 1L,
1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L,
0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L, 1L, 1L,
1L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L,
0L, 0L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L,
0L, 0L, 1L, 1L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 1L),
Species_ID = c("C", "C", "C", "C", "C", "C", "D", "C", "C", "C", "C",
"C", "C", "C", "C", "D", "C", "A", "C", "C", "C", "C", "C", "C",
"C", "C", "D", "C", "C", "C", "C", "C", "C", "D", "D", "C", "A",
"A", "C", "C", "D", "C", "C", "D", "C", "C", "C", "C", "A", "A",
"C", "C", "C", "C", "C", "C", "A", "C", "C", "C", "C", "C", "A",
"D", "A", "C", "C", "A", "C", "C", "C", "C", "C", "C", "C", "C",
"C", "C", "C", "C", "D", "C", "C", "A", "A", "C", "C", "C", "C",
"C", "C", "C", "A", "A", "C", "C", "C", "C", "C", "C", "C", "D",
"C", "C", "A", "C", "C", "C", "C", "C", "D", "C", "C", "C", "C",
"D", "A", "C", "C", "C", "C", "C", "A", "C", "C", "C", "C", "C",
"C", "C", "C", "C", "D", "C", "C", "A", "C", "A", "D", "C", "A",
"C", "C", "D", "C", "C", "C", "D", "C", "C", "C", "C", "C", "C",
"C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C",
"C", "C", "C", "C", "C", "A", "C", "D", "C", "C", "C", "C", "C",
"D", "C", "C", "C", "A", "C", "C", "C", "C", "D", "C", "C", "C",
"A", "C", "A", "C", "C", "D", "A", "C", "A", "C", "A", "C", "A",
"A", "D", "D", "D", "A", "D", "A", "A", "A", "A", "D", "D", "A",
"C", "D", "D", "D", "C", "C", "C", "C", "C", "C", "A", "C", "C",
"C", "C", "C", "C", "C", "C", "A", "C", "C", "C", "C", "C", "A",
"C", "C", "C", "A", "C", "C", "C", "C", "D", "A", "C", "C", "C",
"C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "C", "D", "C",
"C", "C", "C", "C", "A", "A", "A", "D", "A", "C", "C", "A", "A",
"A", "A", "A", "C", "A", "A", "C", "C", "C", "C", "C", "D", "D",
"D", "C", "C", "D", "C", "C", "D", "D", "D", "D", "D", "C", "D",
"D", "C", "C", "C", "C", "D", "C", "C", "D", "A", "D", "D", "C",
"C", "D", "A", "C", "A", "D", "D", "C", "C", "C", "D", "C", "C",
"D", "C", "C", "D", "C", "C", "C", "C", "A", "A", "C", "C", "C",
"C", "C", "D", "D", "C", "C", "C", "C", "C", "A", "D", "C", "D",
"A", "A", "C", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B",
"B", "B", "B", "B", "B", "B", "B", "B", "B", "B", "B")))
data<-as.data.frame(data)
attach(data)
m1<-glmer(Outcome~ Species_ID+(1 |Owner_ID),
data=data, family= (binomial(link="logit")),na.action = na.omit,
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 1000000)))
#get convergence error (tol >0.002)
#Ben Bolker advised to first do this as a check because of the warning:
m1_fit_all<-allFit(m1)
ss<-summary(m1_fit_all)
ss$which.OK
#all seem OK
m1b<-glmer(Outcome~ 1+(1 |Owner_ID),
data=data, family= (binomial(link="logit")),na.action = na.omit,
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 1000000)))
anova(m1,m1b)
#so, an effect of species (as expected - in line with our previous research)
summary(m1)
#z values look a bit suspect (to me)
#splitting dataset by species (A v B first)
# to suss what's going on. Will give you an example of what look like mad odds ratios
a_b_data<-data %>% filter(Species_ID%in% c("A", "B"))
detach(data)
attach(a_b_data)
#and then re-run
m1c<-glmer(Outcome~ Species_ID+(1 |Owner_ID),
data=a_b_data, family= (binomial(link="logit")),na.action = na.omit,
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 1000000)))
#no error this time
m1e<-glmer(Outcome~ 1+(1 |Owner_ID),
data=a_b_data, family= (binomial(link="logit")),na.action = na.omit,
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 1000000)))
anova(m1c,m1e)
#species has an effect
summary(m1c)
#z value looks less crazy, but want to take a look at the odds ratios
se <- sqrt(diag(vcov(m1c)))
(tab <- cbind(Est = fixef(m1c), LL = fixef(m1c) - 1.96 * se, UL = fixef(m1c) + 1.96 *
se))
#this for odds ratios. Upper limit looks especially mad (24,231,520!!)
exp(tab)
#I'll give an example of a normal-looking one too
#splitting dataset by another species pair (C v D)
c_d_data<-data %>% filter(Species_ID%in% c("C", "D"))
detach(a_b_data)
attach(c_d_data)
#and then re-run
m1r<-glmer(Outcome~ Species_ID+(1 |Owner_ID),
data=c_d_data, family= (binomial(link="logit")),na.action = na.omit,
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 1000000)))
#no error
m1t<-glmer(Outcome~ 1+(1 |Owner_ID),
data=c_d_data, family= (binomial(link="logit")),na.action = na.omit,
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 1000000)))
anova(m1r,m1t)
#effect of species here too
#get the odds ratios
se <- sqrt(diag(vcov(m1r)))
(tab <- cbind(Est = fixef(m1r), LL = fixef(m1r) - 1.96 * se, UL = fixef(m1r) + 1.96 *
se))
#this for (less alarming) odds ratios.
exp(tab)
Dr Emma Mellor
Research Associate
University of Bristol
Bristol Veterinary School
Langford House
Langford
BS40 5DU
My working days are generally Monday, Tuesday and Friday. My work schedule may not be the same as yours - please do not feel obliged to respond outside of your own working hours
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models using r-project.org<mailto:R-sig-mixed-models using r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative HTML version deleted]]
More information about the R-sig-mixed-models
mailing list