[R-sig-ME] How to report and quantify the random effect in a logistic model?

[Phillip Alday]

To get the confidence intervals from the random effects, you need to
either use profiling or bootstrapping. Variances -- like the random
effects -- tend to have very skewed sampling distributions, so symmetric
(Wald) confidence intervals based on standard errors don't make sense.

I wouldn't back transform the RE estimates. They are meaningful in their
own right as the variance between groups. For example, in your model,
the random effect for CITY is just the variance of (1.5) between the
intercepts for cities. I would just report the model summary as a table,
but I guess you could also write something like "The standard deviation
of distribution of the intercept between cities was 3.1" but that seems
very awkward to me.

For something like "proportion of variance explained", you're looking
for something like a standardized effect size, but that is *very*
difficult to define in a meaningful way for GLMs and LMMs and thus
doubly so for GLMMs. Henrik Singmann has a nice way to explain the issue
briefly to reviewers
(https://afex.singmann.science/forums/topic/compute-effect-sizes-for-mixed-objects#post-295)
and links to the larger GLMM FAQ question on that section.

Best,
Phillip



On 2/12/20 8:01 pm, Pi wrote:
> Hello.
> 
> I'm fitting a logistic regression model with mixed effects using the
> package glmmTMB. (Because the dataset is quite large and lme4 produces out
> of memory errors, even increasing memory.limit).
> 
> I need help to interpret and report the output.
> 
>     Family: binomial  ( logit )
>     Formula:      OUTPUT ~ SEX + YEAR + OTHER +  (1|CITY/ID)
>     Data: mydata
> 
>           AIC       BIC    logLik  deviance  df.resid
>      890000  891000 -450000  889000    1000000
> 
>     Random effects:
> 
>     Conditional model:
>      Groups Name        Variance Std.Dev.
>      ID:CITY (Intercept) 10.0    3.1
>      CITY    (Intercept) 1.5    1.2
>     Number of obs: 1000009, groups:  ID:CITY, 200000; CITY, 20
> 
>     Conditional model:
>                  Estimate Std.Error z value Pr(>|z|)
>     (Intercept)  0.79   0.28    2.9   0.005
>     SEX1       -0.21   0.017 -12   <2e-16
>     YEAR        0.48   0.0048 100   <2e-16
>     OTHER       -0.70   0.005 -130   <2e-16
> 
> Output from sjPlot:
> 
>     Random Effects
>     σ2 3.29
>     τ00 ID:CITY 10.0
>     τ00 CITY 1.5
>     ICC 0.78
> 
> 
> How can I report the effect of CITY and its confidence interval?
> I think most people would report the CI of the odds of the Intercept using
> the variance to calculate
> 
>      { exp(Intercept-1.96*1.2) , exp(Intercept+1.96*1.2) }
> 
> but this doesn't take into account the standard error of the Intercept from
> the conditional model, 0.28. How should I combine them?
> (I think this is the estimated deviation for the Intercept excluding the
> random effects).
> 
> Is it acceptable to ignore the Intercept value and just say...?
> "The 95% CI for the odds ratio for the CITY is
> {exp(-1.96\*1.2),exp(+1.96\*1.2)}"
> 
> or
> 
> "The odds of OUTPUT is multiplied by a number between exp(-1.96\*1.2) and
> exp(+1.96\*1.2) due to the variability of CITY". I don't know how to
> include here the 95%IC argument.
> 
> I have also tried to calculate the residuals with residuals(model) but it
> produces an out of memory error.
> Is there any way to do it approximately? or is there already some useful
> information in the output?
> 
> I just need to say what proportion of the total variance (or of the
> residuals) is explained by the variable CITY.
> 
> The σ2 value reported by sjPlot is the variance of the residuals.
> 
> What about the variance of ID:CITY (Intercept) 10.0? How should I report
> it? How can I split it into variance due to ID and variance due to the
> interaction of ID and CITY?
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>