[R-meta] I2 interpretation for Multilevel meta-analysis with moderators

[Viechtbauer, Wolfgang (SP)]

Hi Ivan,

I would say forget about I^2 in models with moderators. Nobody gets the meaning of this anyway :)

But in some way, one could interpret the 68.35% as indicating that 68.35% of the remaining variance (that is not accounted for already by the moderators) is due to the sum of study and effect size level heterogeneity (and the remaining 31.65% is due to sampling variance). I find this not particularly useful information.

Personally, as reader, I would want to know:

- for the model without moderators (say res0), the size of the variance components and whether the heterogeneity in the underlying true effects is more due to between-study or within-study heterogeneity; in essence:

res0$sigma2
100 * res0$sigma2 / sum(res0$sigma2)

- one could also report the I^2 values for between-study heterogeneity, within-study heterogeneity, and sampling variance; in essence:

k <- res0$k
wi <- 1/res0$vi
vt <- (k-1) * sum(wi) / (sum(wi)^2 - sum(wi^2))
100 * c(res0$sigma2, vt) / (sum(res0$sigma2) + vt)

- and for the model involving moderators (say res1), how much of the between-study and within-study heterogeneity is accounted for by the moderators; in essence:

100 * pmax(0, (res0$sigma2 - res1$sigma2) / res0$sigma2)

(note: pmax(0, ...) since these pseudo R^2 values can be negative)

So, no I^2 type values based on res1.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Ivan Jukic
>Sent: Tuesday, 12 October, 2021 9:51
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] I2 interpretation for Multilevel meta-analysis with moderators
>
>Dear all,
>
>I'm currently conducting a multilevel meta-analysis with moderators. While the
>interpretation of the I2 for the "regular" multilevel meta-analysis is pretty
>much straightforward, I found it very confusing when moderators are added to the
>mix.
>
>I read, among other resources, the following:
>http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-November/000377.html
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2019-September/001756.html
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000138.html
>
>Even after reading all of this, I'm still a bit unsure about the interpretation.
>
>So, if I run the following:
>
>res <- rma.mv(yi = yi,
>               V = vi,
>               data = data,
>               random = ~ 1 | Study/es_id,
>               method = "REML",
>               test = "t",
>               slab = Study,
>               mods = ~ mod1 + mod2 + mod3)
>
>and obtain total I2 of 68.35% for this model, does this mean that after
>accounting for variance via moderators, the remaining variance unaccounted for
>study and effect size is 68.35%.
>
>I came up with this interpretation while reading one of the posts on this SIG
>(linked above). However, I'm not sure if it's correct, and to be honest,
>completely understand the meaning of "the remaining variance *unaccounted* for
>study and effect size". Therefore, I would really appreciate if someone could
>clarify the interpretation of the I2 for multilevel meta-analysis with
>moderators.
>
>Cheers,
>Ivan