[R-meta] Differences in I^2 estimation between meta and metafor packages

[Mike Cheung]

As a follow-up of the discussion, the metaSEM package implements these
definitions of the I^2 with the likelihood-based CIs on them.

library(metaSEM)

summary( meta(y=yi, v=vi, data=Hox02, I2=c("I2q", "I2hm", "I2am"),
intervals.type="LB") )

Heterogeneity indices (I2) and their 95% likelihood-based CIs:
                                  lbound Estimate ubound
Intercept1: I2 (Q statistic)     0.27625  0.60780 0.6749
Intercept1: I2 (harmonic mean)   0.27713  0.60885 0.6766
Intercept1: I2 (arithmetic mean) 0.26209  0.59052 0.6431

A brief introduction is available here:
https://books.google.com.sg/books?id=Pw7QBwAAQBAJ&pg=PA92&lpg=PA92&dq=%224.3.3+Quantifying+the+degree+of+the+heterogeneity+of+effect+sizes%22&source=bl&ots=zDjRbpiGL5&sig=ACfU3U10nYLvLi9idJ_QnXCKcasi8sDJ3g&hl=en&sa=X&ved=2ahUKEwjk3t6lppziAhXZbSsKHcEmA70Q6AEwAHoECAUQAQ#v=onepage&q=%224.3.3%20Quantifying%20the%20degree%20of%20the%20heterogeneity%20of%20effect%20sizes%22&f=false

-- 
---------------------------------------------------------------------
 Mike W.L. Cheung               Phone: (65) 6516-3702
 Department of Psychology       Fax:   (65) 6773-1843
 National University of Singapore
 http://mikewlcheung.github.io/
<http://courses.nus.edu.sg/course/psycwlm/internet/>
---------------------------------------------------------------------

On Tue, May 14, 2019 at 6:26 PM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Good points. I wrote up that FAQ because this question comes up quite a
> bit, since (by default, that is, with REML estimation) metafor does deviate
> a bit from how I^2 is typically computed (i.e., with 100*(Q - (k-1))/Q), so
> users are sometimes surprised by this.
>
> And I fully agree that there isn't a *right* way of computing I^2, just
> different definitions. I actually find the way the 'typical' sampling
> variance is computed (which is implicitly used by both definitions) rather
> weird. Other definitions have been proposed (in particular, by
> Takkouche/Spiegelman/Crippa), such as:
>
> I^2 = tau^2 / (tau^2 + s^2), where s^2 = k/sum(wi), where wi=1/vi (i.e.,
> the haromic mean of the sampling variances)
>
> or
>
> I^2 = 1/k sum(tau^2 / (vi + tau^2)).
>
> Those actually seem a bit more intuitive to me. The 'hetmeta' package
> provides those versions (called R_I and R_b).
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Guido Schwarzer [mailto:sc using imbi.uni-freiburg.de]
> Sent: Tuesday, 14 May, 2019 11:59
> To: Viechtbauer, Wolfgang (SP); Rushkin, Megan C;
> r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] Differences in I^2 estimation between meta and
> metafor packages
>
> Am 13.05.19 um 10:45 schrieb Viechtbauer, Wolfgang (SP):
>
> > Hi Megan,
> >
> > This is answered here:
> >
> >
> http://www.metafor-project.org/doku.php/faq#how_are_i_2_and_h_2_computed_i
>
> This website nicely describes the differences in the calculation of I2
> in metafor and meta. As this is written from the metafor perspective, I
> would like to describe why meta uses the "other" definition. ;-)
>
> Wolfgang mentions two advantages of the metafor implementation of I2:
> 1) "more general definition"
> 2) "values of I2 and H2 will be consistent with ... tau2"
>
> I agree that the metafor implementation is more general as one gets a
> different I2 value for each estimation method of tau2. However, the meta
> implementation is also based on a generalization (to the situation in
> which precisions differ between studies) - see section 3.3 in Higgins &
> Thompons (2002).
>
> On the other hand, while the metafor implementation guarantees
> consistent estimates for I2 and tau2, the meta implementation guarantees
> consistency of the I2 estimate and the test for heterogeneity (which -
> like I2 - is based on Q and the number of studies).
>
> In summary, one should know the differences in the estimation of I2
> between metafor and meta, however, there is (in my opinion) no clear
> "winner".
>
> Best wishes, Guido
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]