[R-meta] Exact confidence intervals meta-analysis estimate

[Viechtbauer, Wolfgang (NP)]

Hi Stefi,

Depends on what you mean by 'exact' and what kind of model you are using for the analysis. For example, when meta-analyzing a bunch of individual proportions under the assumption of homogeneity, then this is identical to simply adding up all of the event counts in the numerator and the sample sizes in the denominator. Then one can compute a Clopper-Pearson CI based on the aggregated data, which is an 'exact' CI. To illustrate:

#######################################################

dat <- dat.pritz1997
dat

# vtype="AV" to compute the sampling variances under the assumption of homogeneity
dat <- escalc(measure="PR", xi=xi, ni=ni, data=dat, add=0, vtype="AV")
dat

# fit equal-effects model
rma(yi, vi, data=dat, method="EE")

# aggregate counts / sample sizes
agg <- escalc(measure="PR", xi=sum(dat$xi), ni=sum(dat$ni))
summary(agg)

# note that the 'aggregated proportion' and its SE is identical to the one from rma()

# Clopper-Pearson CI
binom.test(sum(dat$xi), sum(dat$ni))

#######################################################

The Wald-type CI from rma() and the C-P CI are quite similar here, but it certainly could also be rather different. Under homogeneity, using an 'exact' logistic regression model for the analysis will also lead to the same pooled estimate, but the CI (computed on the logit scale and then back-transformed) is again a Wald-type CI which will be different from the two CIs above. To illustrate:

#######################################################

res <- rma.glmm(measure="PLO", xi=xi, ni=ni, data=dat, method="EE")
predict(res, transf=transf.ilogit)

#######################################################

Once we go to random-effects models, one can still use rma.glmm(), but the CI is still a Wald-type CI. If you then want a CI that has guaranteed coverage of (at least) 95% (like the C-P CI), then I am not even sure such a method exists.

In case you have not seen this, you might also want to take a look at:

https://www.ncbi.nlm.nih.gov/books/NBK179162/

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Stefi Iaquinto
>Sent: Tuesday, 05 July, 2022 20:22
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Exact confidence intervals meta-analysis estimate
>
>Hi everyone,
>
>I’m currently running a simulation study with a single proportion as outcome.
>As I want to use some “extreme" true proportions (0.95, 0.97 …), I was wondering,
>if there is an option with rma.uni() to calculate exact confidence intervals for
>the overall effect estimate from a meta-analysis (If I’m not mistaken, the
>default is calculating wald-type confidence intervals)?
>
>Thank you!
>
>Best,
>Stefi