[R-meta] I2 in fixed-effect or equal-effects meta-analysis

[Viechtbauer, Wolfgang (NP)]

Indeed, but Yefeng raises the valid question why the output would include I^2 even when method='EE'.

I just decided to include it as a descriptive measure. Let me just focus on H^2 here for a second, which is Q / df (unless we compute it directly from the estimate of tau^2 from a random-effects model, but this is just a small side-issue -- https://www.metafor-project.org/doku.php/faq#how_are_i_2_and_h_2_computed_i). This is also called the Birge ratio and quantifies the departure from the equal-effects assumption in a way that does not depend on the number of studies (recall that E(Q) = df under the equal-effects assumption). So, using method='EE' *assumes* that the true effects are homogeneous, but they may not be. I think it's useful to know whether this assumption is reasonable or not. For this, we have the Q-test, but H^2 is a descriptive measure of the departure from homogeneity. And I^2 is just a transformation of H^2, namely I2 = (H2 - 1) / H2, that somehow become much more population than H^2, so it's also reported.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Mathias Weis Damkjær via R-sig-meta-analysis
> Sent: Thursday, May 22, 2025 06:13
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Mathias Weis Damkjær <mwdamkjaer using health.sdu.dk>
> Subject: Re: [R-meta] I2 in fixed-effect or equal-effects meta-analysis
>
> Dear Yefeng Yang,
>
> "Fixed effects" and "equal effects" meta-analyses have different assumptions.
> Although the analysis and model output is identical, the interpretation or
> "estimand" is different, i.e., the target of inference.
>
> Wolfgang has written a post on the difference here:
> https://wviechtb.github.io/metafor/reference/misc-models.html
>
> I found this paper helpful aswell:
> https://research-
> information.bris.ac.uk/ws/portalfiles/portal/146974606/FixedEffectsPaperRev3.pdf
>
> However, there as several papers on this, and you might find one that you like
> more.
>
> Kind regards,
> Mathias
>
> Mathias Weis Damkjr
> MD, PhD-student
> Cochrane Denmark &
> Centre for Evidence-Based Medicine Odense (CEBMO)
> University of Southern Denmark
>
> T +45 21274558
> E mailto:mwdamkjaer using health.sdu.dk<mailto:dlaursen using health.sdu.dk>
> W http://www.cebmo.dk/<http://www.cebmo.dk/>;
> http://www.cochrane.dk/<http://www.cochrane.dk/>
>
> ________________________________
> From: R-sig-meta-analysis <mailto:r-sig-meta-analysis-bounces using r-project.org> on
> behalf of Yefeng Yang via R-sig-meta-analysis <mailto:r-sig-meta-analysis using r-
> project.org>
> Sent: Thursday, May 22, 2025 5:57:00 AM
> To: mailto:r-sig-meta-analysis using r-project.org <mailto:r-sig-meta-analysis using r-
> project.org>
> Cc: Yefeng Yang <mailto:yefeng.yang1 using unsw.edu.au>
> Subject: [R-meta] I2 in fixed-effect or equal-effects meta-analysis
>
> Dear community,
>
> Recently, I played with the fixed-effect or equal-effects meta-analysis
> implemented in `metafor` package and surprinsly found that the model output
> provides I^2 value. To me this is a bit confusing, because the fixed-effect or
> equal-effects meta-analysis assumes that there is no heterogeneity (no tau^2 and
> so no I2). All observed differences in effect sizes are assumed to be due to
> sampling variance of the effect size estimates. Although one can computationally
> get I2 value from  fixed-effect or equal-effects meta-analysis via 100% * (Q-(k-
> 1))/Q, but what does I2 exactly mean here?
>
> Interestingly, the I2 provided by equal-effects meta-analysis is exactly the
> same as random-effects meta-analysis. The so-called fixed-effects meta-analysis
> (the true effect sizes are fixed across k studies rather than randomly drawn
> from a population) also gives I2.
>
> The reproducible example:
> library(metafor)
> dat <- dat.bangertdrowns2004
> # fixed-effect
> res <- rma(yi, vi, data=dat, method="EE"), with the output:
> Equal-Effects Model (k = 48)
>
> I^2 (total heterogeneity / total variability):   56.12%
> H^2 (total variability / sampling variability):  2.28
>
> # random-effects
> res2 <- rma(yi, vi, data=dat)