[R-sig-ME] Predicted probabilites with CIs for multilevel logistic regression with prior weights

[Sam Crawley]

Hi Daniel,

Thanks very much, that clears it up.

Cheers,
Sam.

On Tue, 18 Jun 2019, at 18:30, d.luedecke using uke.de wrote:
> > However, should I be using the type = "re" parameter?
> 
> This depends on whether you would like to have confidence or prediction 
> intervals. I'm not sure if there's a clear definition, or a consensus 
> on how to obtain prediction intervals. There's a section about this in 
> Ben Bolker's GLMM-FAQ:
> http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#predictions-andor-confidence-or-prediction-intervals-on-predictions
> 
> He states that you want to add the residual variance to compute 
> prediction intervals. However, there are also some caveats: 
> https://stackoverflow.com/questions/14358811/extract-prediction-band-from-lme-fit
> 
> Ben also has a comment in his examples: "## must be adapted for more 
> complex models". I'm not sure if this refers only to make sure to 
> compute sigma properly, or if you also need to take the random effect 
> variances into account. ggeffects currently computes the standard error 
> for type = "re" based on the *random effects variances* (see details 
> here: https://easystats.github.io/insight/reference/get_variance.html). 
> This seems to be even a bit more conservative than just taking the 
> residual variance. But again, any comments on this are welcome. I've 
> been in email exchange with Russell Lenth, maintainer of the 
> emmeans-package, and we were also talking about if it's actually 
> straightforward to obtain prediction intervals, or not. In the latest 
> emmeans-release, there is a vignette on this topic 
> (https://cran.r-project.org/web/packages/emmeans/vignettes/predictions.html).
> 
> In short: you don't need to specify type = "re" if you want to get 
> predictions from your mixed models. Predictions for both type = "fe" 
> and type = "re" are always on a population-level. However, you need 
> type = "re" if you want to make predictions at each level of the random 
> effects (group factor), see 
> https://strengejacke.github.io/ggeffects/articles/randomeffects.html#marginal-effects-for-each-level-of-random-effects (though here you don't get any intervals).
> 
> Best
> Daniel
> 
> -----Ursprüngliche Nachricht-----
> Von: Sam Crawley <sam_crawley using warpmail.net> 
> Gesendet: Dienstag, 18. Juni 2019 03:44
> An: d.luedecke <d.luedecke using uke.de>; r-sig-mixed-models using r-project.org
> Betreff: Re: [R-sig-ME] Predicted probabilites with CIs for multilevel 
> logistic regression with prior weights
> 
> Hi again Daniel (and list),
> 
> Thanks again for the below. I have been using the ggpredict() function, 
> and it works well. However, should I be using the type = "re" 
> parameter? Or is this only required when attempting to predict values 
> for each group? (I have read the ggeffects documentation on this, but 
> it's still not entirely clear to me).
> 
> When adding type="re", the confidence intervals become very wide, which 
> is obviously not ideal.
> 
> Thanks,
> Sam Crawley.
> 
> On Tue, 11 Jun 2019, at 03:30, d.luedecke using uke.de wrote:
> > Hi Sam,
> > 
> > you could the "ggeffects" package
> > (https://strengejacke.github.io/ggeffects/), and there is also an example
> > for a logistic mixed effects model
> > (https://strengejacke.github.io/ggeffects/articles/practical_logisticmixedmo
> > del.html), which might help you.
> > 
> > For binomial models, using weights often results in the following warning:
> > #> non-integer #successes in a binomial glm!
> > 
> > However, CIs for the predicted probabilities can be calculated nevertheless
> > (at least in my quick example). Note that afaik, mixed models in R do
> > correctly not account for sampling weights. However, Thomas Lumley, author
> > of the survey-package, works on a survey-function for mixed models
> > (https://github.com/tslumley/svylme), probably the GitHub version is quite
> > stable (haven't tested yet).
> > 
> > An alternative would be the "scale_weights()" function from the
> > sjstats-package
> > (https://strengejacke.github.io/sjstats/articles/mixedmodels-statistics.html
> > #rescale-model-weights-for-complex-samples ), which rescales sampling
> > weights so they can be used as "weights" for the mixed models function you
> > have in R (lme4, lme, ...).
> > 
> > Based on that function, I have a small example that demonstrates how to
> > compute predicted probabilities for mixed models with (sampling) weights
> > (ignore the warnings, this is just for demonstration purposes):
> > 
> > library(lme4)
> > library(sjstats) # for scale_weights() and sample data
> > library(ggeffects) # for ggpredict()
> > 
> > data(nhanes_sample)
> > set.seed(123)
> > nhanes_sample$bin <- rbinom(nrow(nhanes_sample), 1, prob = .3)
> > nhanes_sample <- scale_weights(nhanes_sample, SDMVSTRA, WTINT2YR)
> > 
> > m <- glmer(
> >   bin ~ factor(RIAGENDR) * age + factor(RIDRETH1) + (1 | SDMVPSU),
> >   family = binomial(),
> >   data = nhanes_sample,
> >   weights = svywght_a
> > )
> > 
> > ggpredict(m, c("age", "RIAGENDR")) %>% plot()
> > 
> > 
> > Best
> > Daniel
> > 
> > -----Ursprüngliche Nachricht-----
> > Von: R-sig-mixed-models <r-sig-mixed-models-bounces using r-project.org> Im
> > Auftrag von Sam Crawley
> > Gesendet: Montag, 10. Juni 2019 10:36
> > An: r-sig-mixed-models using r-project.org
> > Betreff: [R-sig-ME] Predicted probabilites with CIs for multilevel logistic
> > regression with prior weights
> > 
> > Hello all,
> > 
> > I am doing a multilevel binomial logistic regression using lme4, and the
> > survey data I am using requires weights to be used. I would like to
> > calculate various predicted probabilities with confidence intervals based on
> > the estimated model. The predict function obviously doesn't give me standard
> > errors, and the recommended method to get these is to use the bootMer
> > function.
> > 
> > However, in my case, the weights provided are not integers, and the bootMer
> > function exits with an error if the weights are not integers (I raised a
> > GitHub issue about this, and was pointed to this list:
> > https://github.com/lme4/lme4/issues/524 ).
> > 
> > So my question is, what is the best way to calculate the predicted
> > probabilities (with confidence intervals) in my case?
> > 
> > I would appreciate any help you can give me, and I'm happy to provide more
> > details if required.
> > 
> > Thanks,
> > Sam Crawley.
> > 
> > _______________________________________________
> > R-sig-mixed-models using r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> > 
> > --
> > 
> > _____________________________________________________________________
> > 
> > Universitätsklinikum Hamburg-Eppendorf; Körperschaft des öffentlichen 
> > Rechts; Gerichtsstand: Hamburg | www.uke.de
> > Vorstandsmitglieder: Prof. Dr. Burkhard Göke (Vorsitzender), Prof. Dr. 
> > Dr. Uwe Koch-Gromus, Joachim Prölß, Marya Verdel
> > _____________________________________________________________________
> > 
> > SAVE PAPER - THINK BEFORE PRINTING
> > 
> >
> 
> --
> 
> _____________________________________________________________________
> 
> Universitätsklinikum Hamburg-Eppendorf; Körperschaft des öffentlichen 
> Rechts; Gerichtsstand: Hamburg | www.uke.de
> Vorstandsmitglieder: Prof. Dr. Burkhard Göke (Vorsitzender), Prof. Dr. 
> Dr. Uwe Koch-Gromus, Joachim Prölß, Marya Verdel
> _____________________________________________________________________
> 
> SAVE PAPER - THINK BEFORE PRINTING
>