[R-meta] Metareg: Significant reduction in tau-squared but no change in i-squared

[Richard Simonson]

Thank you, Dr. Wolfganag, this explanation was very helpful!

23% was a typo on my end. I meant to say 12%

On Thu, Sep 3, 2020 at 5:06 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Richard,
>
> See comments below.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org]
> >On Behalf Of Richard Simonson
> >Sent: Wednesday, 02 September, 2020 18:48
> >To: r-sig-meta-analysis using r-project.org
> >Subject: [R-meta] Metareg: Significant reduction in tau-squared but no
> >change in i-squared
> >
> >I've conducted a meta-regression with metareg on a random-effects
> >meta-analysis and am attempting to describe the results. The output of the
> >model is at the end of my description
> >
> >Based on the regression model, I'm given that our R-squared is 12%, which
> I
> >take that to mean ~23% of the heterogeneity was accounted for when
> >covarying the model.
>
> Not sure where you get the 23% from. The R^2 of 12% means that (an
> estimated) 12% of the heterogeneity was accounted for (by the moderator(s)
> included in the model).
>
> >I also see that there was a reduction in tau-squared from the random
> >effects meta-analysis to the meta-regression, but no change in I-squared.
> >
> >Can someone help me understand why, with a positive non-zero r-squared,
> >there's a change in tau-squared but not in I-squared?
>
> First of all, all these things (I^2 in a model without moderators, I^2 in
> a meta-regression model, and R^2) are estimates, so they can simply be
> 'off'. For example, it can happen that tau^2 or I^2 actually increase when
> including a moderator in a model, although in theory that doesn't make
> sense.
>
> Also, I^2 in a meta-regression model has a different interpretation than
> in a model without moderators. In a model without moderators, I^2 is an
> estimate how much of the total variability (which is composed of
> heterogeneity and sampling variability) is due to heterogeneity. However,
> in a meta-regression model, I^2 is an estimate how much of the *unaccounted
> for variability* (which is composed of residual / unaccounted for
> heterogeneity and sampling variability) is due to the residual /
> unaccounted for heterogeneity. That's different than asking how much of the
> *total variability* is due to residual / unaccounted for heterogeneity. So
> I^2 in a meta-regression model is a bit of a strange statistic anyway and I
> am not sure how many people actually grasp its correct interpretation.
>
> As a more technical point: The way I^2 is computed in meta-regression
> models is also a bit strange. The way the 'typical' sampling variance is
> computed under such models actually depends on the moderators included in
> the model, which one could argue is a bit odd. But given that people
> typically compute I^2 with (Q-df)/df (where Q could be the test for
> heterogeneity or the test for residual heterogeneity) this is what happens.
>
> >Which one is more feasible to report that we were able to account for some
> >of the heterogeneity?
>
> In meta-regression models, I would report R^2 and not report I^2. You can
> also report the test for residual heterogeneity, so if somebody feels the
> need to compute I^2 based on that, they always can.
>
> >Thanks!
> >
> >Meta output:
> >I-squared: 87%
> >tau-squared: .27
> >
> >Meta-reg output
> >tau^2 (estimated amount of residual heterogeneity):     0.2426 (SE =
> 0.2570)
> >tau (square root of estimated tau^2 value):                    0.4925
> >I^2 (residual heterogeneity / unaccounted variability):    86.68%
> >H^2 (unaccounted variability / sampling variability):       7.51
> >R^2 (amount of heterogeneity accounted for):               12.25%
>


-- 

Thank you,

Richard J. Simonson

HFES ERAU Chapter President

PhD Candidate

Department of Human Factors and Behavioral Neurobiology

Embry-Riddle Aeronautical University

	[[alternative HTML version deleted]]