[R-meta] Quantify the amount of residual heterogeneity using the QE value

[Viechtbauer, Wolfgang (NP)]

Dear Max,

Whether we call a variance component tau^2 or sigma^2 or chicken^2 really doesn't matter. When I fit a standard random-effects model using rma.mv() as shown here:

https://www.metafor-project.org/doku.php/analyses:konstantopoulos2011

or here:

https://www.metafor-project.org/doku.php/analyses:crede2010

then the variance component (often denoted tau^2 in the literature, but not always so) which reflects the amount of heterogeneity in the true effect sizes is denoted sigma^2 in the output. It's still the same parameter.

In the multilevel model, there are two variance components (denoted sigma^2.1 and sigma^2.2), one for between-study (or between-DOI) heterogeneity and one for within-study (or within-DOI) heterogeneity. So, if you want to say something about the amount of heterogeneity accounted for by the moderators, then you can do the same thing that we do for a standard random/mixed-effects meta-regression model, namely compute the proportional reduction in the variance, except that we can now do this for each variance component separately:

(no_mods$sigma2 - all_mods$sigma2) / no_mods$sigma2

You can also compute how much the *total* amount of heterogeneity (between + within) is reduced:

(sum(no_mods$sigma2) - sum(all_mods$sigma2)) / sum(no_mods$sigma2)

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of max doering via R-sig-meta-analysis
> Sent: Thursday, March 20, 2025 13:48
> To: r-sig-meta-analysis using r-project.org
> Cc: max doering <1maxdoering using gmail.com>
> Subject: [R-meta] Quantify the amount of residual heterogeneity using the QE
> value
>
> Dear R-sig-meta-analysis community,
>
> I am currently conducting a meta analysis using metafor.
> For this, I want to compare a model with no moderators and one with
> all moderators:
>
> no_mods = rma.mv(yi, vi, random = ~1|DOI/individual_level, data = data)
> all_mods = rma.mv(yi, vi, random = ~1|DOI/individual_level, mods =
> ~mod1 + mod2 + ..., data = data)
>
> I would like to answer the question, how much of the residual
> heterogeneity in the no_mod model can be explained by adding all the
> moderators. In the end, I want to say something like "the moderators
> account for X% of the residual heterogeneity".
>
> Unfortunately, the model does not calculate a tau^2 value (at least it
> is 0 in both models), so my thought was that I could maybe use the QE
> value for a calculation like:
>
> (1 - ((all_mods $QE/all_mods $QEdf)/(no_mods $QE/no_mods $QEdf)))*100
>
> basically calculating the change of the QE value in percent with
> respect to their dfs.
>
> In summary, my questions are:
> 1) Can I use the QE value to quantify the amount of residual
> heterogeneity accounted for by all moderators?
> 2) If the answer is no or if there is a simpler solution, what options
> are there?
>
> Thank you and best regards,
> Max