[R-meta] Differences in I^2 estimation between meta and metafor packages

[Viechtbauer, Wolfgang (SP)]

Good points. I wrote up that FAQ because this question comes up quite a bit, since (by default, that is, with REML estimation) metafor does deviate a bit from how I^2 is typically computed (i.e., with 100*(Q - (k-1))/Q), so users are sometimes surprised by this.

And I fully agree that there isn't a *right* way of computing I^2, just different definitions. I actually find the way the 'typical' sampling variance is computed (which is implicitly used by both definitions) rather weird. Other definitions have been proposed (in particular, by Takkouche/Spiegelman/Crippa), such as:

I^2 = tau^2 / (tau^2 + s^2), where s^2 = k/sum(wi), where wi=1/vi (i.e., the haromic mean of the sampling variances)

or

I^2 = 1/k sum(tau^2 / (vi + tau^2)).

Those actually seem a bit more intuitive to me. The 'hetmeta' package provides those versions (called R_I and R_b).

Best,
Wolfgang

-----Original Message-----
From: Guido Schwarzer [mailto:sc using imbi.uni-freiburg.de] 
Sent: Tuesday, 14 May, 2019 11:59
To: Viechtbauer, Wolfgang (SP); Rushkin, Megan C; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Differences in I^2 estimation between meta and metafor packages

Am 13.05.19 um 10:45 schrieb Viechtbauer, Wolfgang (SP):

> Hi Megan,
>
> This is answered here:
>
> http://www.metafor-project.org/doku.php/faq#how_are_i_2_and_h_2_computed_i

This website nicely describes the differences in the calculation of I2 
in metafor and meta. As this is written from the metafor perspective, I 
would like to describe why meta uses the "other" definition. ;-)

Wolfgang mentions two advantages of the metafor implementation of I2:
1) "more general definition"
2) "values of I2 and H2 will be consistent with ... tau2"

I agree that the metafor implementation is more general as one gets a 
different I2 value for each estimation method of tau2. However, the meta 
implementation is also based on a generalization (to the situation in 
which precisions differ between studies) - see section 3.3 in Higgins & 
Thompons (2002).

On the other hand, while the metafor implementation guarantees 
consistent estimates for I2 and tau2, the meta implementation guarantees 
consistency of the I2 estimate and the test for heterogeneity (which - 
like I2 - is based on Q and the number of studies).

In summary, one should know the differences in the estimation of I2 
between metafor and meta, however, there is (in my opinion) no clear 
"winner".

Best wishes, Guido