[R-meta] Metareg: Significant reduction in tau-squared but no change in i-squared

[Viechtbauer, Wolfgang (SP)]

Dear Richard,

See comments below.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On Behalf Of Richard Simonson
>Sent: Wednesday, 02 September, 2020 18:48
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Metareg: Significant reduction in tau-squared but no
>change in i-squared
>
>I've conducted a meta-regression with metareg on a random-effects
>meta-analysis and am attempting to describe the results. The output of the
>model is at the end of my description
>
>Based on the regression model, I'm given that our R-squared is 12%, which I
>take that to mean ~23% of the heterogeneity was accounted for when
>covarying the model.

Not sure where you get the 23% from. The R^2 of 12% means that (an estimated) 12% of the heterogeneity was accounted for (by the moderator(s) included in the model).

>I also see that there was a reduction in tau-squared from the random
>effects meta-analysis to the meta-regression, but no change in I-squared.
>
>Can someone help me understand why, with a positive non-zero r-squared,
>there's a change in tau-squared but not in I-squared?

First of all, all these things (I^2 in a model without moderators, I^2 in a meta-regression model, and R^2) are estimates, so they can simply be 'off'. For example, it can happen that tau^2 or I^2 actually increase when including a moderator in a model, although in theory that doesn't make sense.

Also, I^2 in a meta-regression model has a different interpretation than in a model without moderators. In a model without moderators, I^2 is an estimate how much of the total variability (which is composed of heterogeneity and sampling variability) is due to heterogeneity. However, in a meta-regression model, I^2 is an estimate how much of the *unaccounted for variability* (which is composed of residual / unaccounted for heterogeneity and sampling variability) is due to the residual / unaccounted for heterogeneity. That's different than asking how much of the *total variability* is due to residual / unaccounted for heterogeneity. So I^2 in a meta-regression model is a bit of a strange statistic anyway and I am not sure how many people actually grasp its correct interpretation.

As a more technical point: The way I^2 is computed in meta-regression models is also a bit strange. The way the 'typical' sampling variance is computed under such models actually depends on the moderators included in the model, which one could argue is a bit odd. But given that people typically compute I^2 with (Q-df)/df (where Q could be the test for heterogeneity or the test for residual heterogeneity) this is what happens.

>Which one is more feasible to report that we were able to account for some
>of the heterogeneity?

In meta-regression models, I would report R^2 and not report I^2. You can also report the test for residual heterogeneity, so if somebody feels the need to compute I^2 based on that, they always can.

>Thanks!
>
>Meta output:
>I-squared: 87%
>tau-squared: .27
>
>Meta-reg output
>tau^2 (estimated amount of residual heterogeneity):     0.2426 (SE = 0.2570)
>tau (square root of estimated tau^2 value):                    0.4925
>I^2 (residual heterogeneity / unaccounted variability):    86.68%
>H^2 (unaccounted variability / sampling variability):       7.51
>R^2 (amount of heterogeneity accounted for):               12.25%