[R-meta] Metareg: Significant reduction in tau-squared but no change in i-squared
Viechtbauer, Wolfgang (SP)
wo||g@ng@v|echtb@uer @end|ng |rom m@@@tr|chtun|ver@|ty@n|
Thu Sep 3 11:06:37 CEST 2020
See comments below.
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On Behalf Of Richard Simonson
>Sent: Wednesday, 02 September, 2020 18:48
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Metareg: Significant reduction in tau-squared but no
>change in i-squared
>I've conducted a meta-regression with metareg on a random-effects
>meta-analysis and am attempting to describe the results. The output of the
>model is at the end of my description
>Based on the regression model, I'm given that our R-squared is 12%, which I
>take that to mean ~23% of the heterogeneity was accounted for when
>covarying the model.
Not sure where you get the 23% from. The R^2 of 12% means that (an estimated) 12% of the heterogeneity was accounted for (by the moderator(s) included in the model).
>I also see that there was a reduction in tau-squared from the random
>effects meta-analysis to the meta-regression, but no change in I-squared.
>Can someone help me understand why, with a positive non-zero r-squared,
>there's a change in tau-squared but not in I-squared?
First of all, all these things (I^2 in a model without moderators, I^2 in a meta-regression model, and R^2) are estimates, so they can simply be 'off'. For example, it can happen that tau^2 or I^2 actually increase when including a moderator in a model, although in theory that doesn't make sense.
Also, I^2 in a meta-regression model has a different interpretation than in a model without moderators. In a model without moderators, I^2 is an estimate how much of the total variability (which is composed of heterogeneity and sampling variability) is due to heterogeneity. However, in a meta-regression model, I^2 is an estimate how much of the *unaccounted for variability* (which is composed of residual / unaccounted for heterogeneity and sampling variability) is due to the residual / unaccounted for heterogeneity. That's different than asking how much of the *total variability* is due to residual / unaccounted for heterogeneity. So I^2 in a meta-regression model is a bit of a strange statistic anyway and I am not sure how many people actually grasp its correct interpretation.
As a more technical point: The way I^2 is computed in meta-regression models is also a bit strange. The way the 'typical' sampling variance is computed under such models actually depends on the moderators included in the model, which one could argue is a bit odd. But given that people typically compute I^2 with (Q-df)/df (where Q could be the test for heterogeneity or the test for residual heterogeneity) this is what happens.
>Which one is more feasible to report that we were able to account for some
>of the heterogeneity?
In meta-regression models, I would report R^2 and not report I^2. You can also report the test for residual heterogeneity, so if somebody feels the need to compute I^2 based on that, they always can.
>tau^2 (estimated amount of residual heterogeneity): 0.2426 (SE = 0.2570)
>tau (square root of estimated tau^2 value): 0.4925
>I^2 (residual heterogeneity / unaccounted variability): 86.68%
>H^2 (unaccounted variability / sampling variability): 7.51
>R^2 (amount of heterogeneity accounted for): 12.25%
More information about the R-sig-meta-analysis