[R-meta] Question regarding metarate calling the Poisson model for meta-analysis

[Viechtbauer, Wolfgang (SP)]

Dear Roel,

If you are looking at cumulative proportions, then I would personally just use a binomial/logistic model. If studies differ in terms of the time periods (e.g., one study provides the cumulative proportion at 30 days while another at 90 days), then this could be accounted for via a moderator in a logistic mixed-effects meta-regression model. rma.glmm() with measure="PLO" can do this.

If you have multiple proportions for the same group of individuals at various follow-up timepoints, then things get more tricky. First of all, when you say "cumulative proportion at 30 days, at 90 days", do you mean that the first proportion is for number of cases between 0-30 days and the second proportion is for the number of cases (among the non-cases left over after 30 days) between 30-90 days? Or do you mean that the first proportion is for the number of cases between 0-30 days and the second proportion is for the number of cases between 0-90 days?

In any case (pun intended), these proportions are not independent of each other. Some relevant articles:

Trikalinos, T. A. & Olkin, I. (2012). Meta-analysis of effect sizes reported at multiple time points: A multivariate approach. Clinical Trials, 9(5), 610-620. https://doi.org/10.1177/1740774512453218 

Trikalinos, T. A. & Olkin, I. (2008). A method for the meta-analysis of mutually exclusive binary outcomes. Statistics in Medicine, 27(21), 4279-4300. https://doi.org/10.1002/sim.3299 

You will find this dataset and the corresponding analysis here:

https://wviechtb.github.io/metadat/reference/dat.fine1993.html

This is actually a slightly more complex case where the outcome is the (log) odds ratio contrasting two conditions. The same principle applies for single groups, except that the outcome would then just me the log odds for individual groups. Note that a 'normal' model is used in this analysis, not a logistic mixed-effects model (for the latter, you would need the raw data).

Best,
Wolfgang

>-----Original Message-----
>From: Willems, R.P.J. (Roel) [mailto:r.willems using amsterdamumc.nl]
>Sent: Monday, 21 March, 2022 14:24
>To: Viechtbauer, Wolfgang (SP); 'r-sig-meta-analysis using r-project.org'
>Subject: RE: Question regarding metarate calling the Poisson model for meta-
>analysis
>
>Dear Prof. Viechtbauer,
>
>Thanks for your prompt reply. Your explanation was very helpful. However, to
>better clarify my previous question: We aim to pool cumulative proportions (No.
>of new events during the specified period / Size of population at start of
>period). We do not aim to pool incidence density (or incidence rate) or incidence
>rate ratios (IRRs) and we do not use the summed individual person-time for the
>population of study.
>
>For pooling the cumulative proportions, we do want to take into account the fact
>that studies used different time periods for which followed the baseline
>population at risk. To do so, we tried to use the mixed-Poisson model with a
>random effect for the different time periods (for example, cumulative proportion
>at 30 days, at 90 days, etc). This has been reported for incidence rate ratios,
>but have not found a way to do this for cumulative proportions (Bagos, Pantelis G
>and Nikolopoulos, Georgios K. "Mixed-Effects Poisson Regression Models for Meta-
>Analysis of Follow-Up Studies with Constant or Varying Durations" The
>International Journal of Biostatistics, vol. 5, no. 1, 2009.
>https://doi.org/10.2202/1557-4679.1168).
>
>Our data already consist of a cumulative proportion with their respective exact
>95% Poisson CI for each study.
>
>Best,
>Roel
>
>-----Oorspronkelijk bericht-----
>Van: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Verzonden: vrijdag 18 maart 2022 11:44
>Aan: Willems, R.P.J. (Roel) <r.willems using amsterdamumc.nl>; 'r-sig-meta-analysis using r-
>project.org' <r-sig-meta-analysis using r-project.org>
>Onderwerp: RE: Question regarding metarate calling the Poisson model for meta-
>analysis
>
>Dear Roel,
>
>I can't quite follow exactly what you are tying to do or have done.
>
>"We wanted to use the random effects, to account for different cumulative time
>periods of follow-up." This sounds like you want some kind of model that includes
>multiple random effects, at multiple levels (e.g., a random effect at the
>estimate level (for overdispersion) and a random effect at some higher level to
>account for clustering / a multilevel structure). rma.glmm() is not set up for
>that. In this case, you can just use glmer() from lme4 directly.
>
>"The pooled results however, showed confidence limits that differed significantly
>when we compared these to exact Poisson 95% CIs that we calculated separately."
>Are you talking about CIs for the individual rates or for the pooled result? If
>you already have an exact CI for the pooled result, then why bother with
>metarate(), rma.glmm(), or glmer()? But if you are talking about the individual
>rates: The CIs are constructed are typically Wald-type CIs (possibly on a
>transformed scale and then back-transformed). So these can differ from exact CIs
>computed directly based on a Poisson distribution. For example:
>
>summary(escalc(measure="IR", xi=10, ti=500))[c("ci.lb", "ci.ub")]
>summary(escalc(measure="IRLN", xi=10, ti=500), transf=exp)[c("ci.lb", "ci.ub")]
>summary(escalc(measure="IRS", xi=10, ti=500), transf=\(x) x^2)[c("ci.lb",
>"ci.ub")]
>
>And comparing those against the exact CI:
>
>round(c(poisson.test(10, 500)$conf.int), 4)
>
>Note that these CIs are typically just used in a forest plot for visualization of
>the individual studies/estimates. They are not used in the actual meta-analysis.
>So even though the CIs may not be 'exact', they do convey approximately the
>uncertainty in the different estimates and how these uncertainties differ from
>each other.
>
>If you prefer to show exact CIs in a forest plot, you can do that as follows:
>
>dat <- dat.nielweise2008
>dat <- escalc(measure="IRLN", xi=x1i, ti=t1i, data=dat, slab=paste0(authors, ",
>", year))
>
>res <- rma(yi, vi, data=dat)
>
>par(mfrow=c(1,2))
>
>forest(res, refline=NA, transf=exp, digits=3, psize=1, header=TRUE)
>
># compute the 'exact' CIs
>cis <- tapply(dat, dat$study, FUN = \(x) poisson.test(x$x1i, x$t1i)$conf.int) cis
><- do.call(rbind, cis)
>
># set up the forest plot using the exact CIs for the individual rates dat <-
>escalc(measure="IR", xi=x1i, ti=t1i, data=dat, slab=paste0(authors, ", ", year))
>forest(dat$yi, ci.lb=cis[,1], ci.ub=cis[,2], refline=NA, digits=3, psize=1,
>header=TRUE, ylim=c(-1.5,res$k+3))
>abline(h=0)
>
># add the estimate from the model
>pred <- predict(res, transf=exp)
>addpoly(pred$pred, ci.lb=pred$ci.lb, ci.ub=pred$ci.ub, row=-1)
>
>I put the two forest plots side-by-side to illustrate how they differ. Note that
>one cannot construct the forest plot on some transformed scale such that the CIs
>will be symmetric when using the exact CIs, so you will end up with these skewed-
>looking CIs, as opposed to:
>
>forest(res, refline=NA, atransf=exp, digits=3, psize=1, header=TRUE)
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: R-sig-meta-analysis
>>[mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of
>>Willems, R.P.J. (Roel)
>>Sent: Friday, 18 March, 2022 9:23
>>To: 'r-sig-meta-analysis using r-project.org'
>>Subject: [R-meta] Question regarding metarate calling the Poisson model
>>for meta- analysis
>>
>>Dear all,
>>
>>We are conducting a systematic review and meta-analysis to determine
>>the risk of infection in previously-colonized patients with
>>multidrug-resistant microorganisms. Using metarate, we have tried to
>>call the random intercept Poisson regression model (Stijnen et al.,
>>2010) from the R package metafor in order to pool cumulative incidence
>>(proportions). We wanted to use the random effects, to account for
>>different cumulative time periods of follow-up. The pooled results
>>however, showed confidence limits that differed significantly when we
>>compared these to exact Poisson 95% CIs that we calculated separately.
>>Is it possible to use your R package to perform meta-analyses using the
>>Poisson-based approach, with a random effect for follow-up time to
>>account for variations in cumulative time period per individual study.
>>We aimed to yield point estimates and 95% CIs for the cumulative incidence (not
>using any risk ratios or comparator groups).
>>
>>We would like to inquire whether it is possible to conduct the intended
>>meta- analysis using this package? Otherwise, could you inform us on a
>>good alternative approach?
>>
>>Best,
>>Roel
>>
>>R.P.J. Willems MD
>>Medical Microbiology and Infection Prevention AII | Location AMC |
>>L1-247 | Meibergdreef 9, 1105 HZ Amsterdam
>>E: r.willems using amsterdamumc.nl<mailto:r.willems using amsterdamumc.nl>
>>www.amsterdamumc.nl<http://www.amsterdamumc.nl> |
>>www.vumc.nl<http://www.vumc.nl> | www.amc.nl<http://www.amc.nl>