# [R-meta] confidence interval for I^2 in multilevel model

Viechtbauer Wolfgang (SP) wolfgang.viechtbauer at maastrichtuniversity.nl
Wed Aug 23 22:28:05 CEST 2017

```Hi Kyle,

With confint(), you can get CIs for variance components. You can plug the CI bounds for a variance component into the equation for I^2 to obtain a CI for I^2. Here is an example:

dat <- get(data(dat.konstantopoulos2011))

res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
res

sav <- confint(res)
sav

W <- diag(1/dat\$vi)
X <- model.matrix(res)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * res\$sigma2 / (res\$sigma2 + (res\$k-res\$p)/sum(diag(P))) ### district and school level I^2
100 * sav[]\$random[1,2:3] / (sav[]\$random[1,2:3] + (res\$k-res\$p)/sum(diag(P))) ### CI for district-level I^2
100 * sav[]\$random[1,2:3] / (sav[]\$random[1,2:3] + (res\$k-res\$p)/sum(diag(P))) ### CI for the school-level I^2

100 * sum(res\$sigma2) / (sum(res\$sigma2) + (res\$k-res\$p)/sum(diag(P))) ### total I^2

But you cannot use this approach to get a CI for the total I^2. For this, you have to use the 'multivariate parameterization' of this model (see: http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011). So:

res <- rma.mv(yi, vi, random = ~ factor(study) | district, data=dat)
res

sav <- confint(res)
sav

W <- diag(1/dat\$vi)
X <- model.matrix(res)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * res\$tau2 / (res\$tau2 + (res\$k-res\$p)/sum(diag(P))) ### total I^2
100 * sav\$random[1,2:3] / (sav\$random[1,2:3] + (res\$k-res\$p)/sum(diag(P))) ### CI for the total I^2

But you are using 'random = ~ 1 | ID', which does not give you a multilevel model. Are you maybe making this mistake?

http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011#a_common_mistake_in_the_three-level_model

Also, 'struct' is only relevant when you have something like a 'random = ~ inner | outer' structure, so struct="DIAG" has no effect.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Porter, Kyle
Sent: Wednesday, August 23, 2017 22:09
To: 'r-sig-meta-analysis at r-project.org'
Subject: [R-meta] confidence interval for I^2 in multilevel model

Hello all,

I am using metafor to calculate an I^2 analogue for a multilevel random effects meta-regression as described at http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate

I would like help in calculating a 95% confidence interval for this derivation of I^2.

Specifically, I am using the code taken below from the website to generalize the concept of I^2 to a clustered/multilevel model.

(the text below is taken from http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate):
Based on the discussion above, it is now very easy to generalize the concept of I2 to such a model (see also Nakagawa & Santos, 2012). That is, we can first compute:

W <- diag(1/dat\$vi)
X <- model.matrix(res)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * sum(res\$sigma2) / (sum(res\$sigma2) + (res\$k-res\$p)/sum(diag(P)))

My specific application is as follows (the outcome is a single proportion):

PPV_prevdat <- escalc(measure="PAS", data=PPV_prev, xi=TP, ni=posTest)

mvregAll <- rma.mv(data=PPV_prevdat, yi, vi, method="REML", random = ~ 1 | ID, struct="DIAG", mods = ~ factor(Bethesda) + factor(Blinded) + prev +factor(NRAS12_13) + factor(HRAS12_13) + factor(KRAS61))

w <- diag(1/PPV_prevdat\$vi)
x <- model.matrix(mvregAll)
P <- w - w %*% x %*% solve(t(x) %*% w %*% x) %*% t(x) %*% w
100 * sum(mvregAll\$sigma2) / (sum(mvregAll\$sigma2) + (mvregAll\$k-mvregAll\$p)/sum(diag(P)))

Any help/suggestion is appreciated.

Thank you,
Kyle Porter

```

More information about the R-sig-meta-analysis mailing list