[R-meta] Quantify the amount of residual heterogeneity using the QE value

[max doering]

Dear Wolfgang,
thank you so much!
You should really consider implementing a parameter called chicken^2...
Best,
Max

Am Do., 20. März 2025 um 14:24 Uhr schrieb Viechtbauer, Wolfgang (NP)
<wolfgang.viechtbauer using maastrichtuniversity.nl>:
>
> Dear Max,
>
> Whether we call a variance component tau^2 or sigma^2 or chicken^2 really doesn't matter. When I fit a standard random-effects model using rma.mv() as shown here:
>
> https://www.metafor-project.org/doku.php/analyses:konstantopoulos2011
>
> or here:
>
> https://www.metafor-project.org/doku.php/analyses:crede2010
>
> then the variance component (often denoted tau^2 in the literature, but not always so) which reflects the amount of heterogeneity in the true effect sizes is denoted sigma^2 in the output. It's still the same parameter.
>
> In the multilevel model, there are two variance components (denoted sigma^2.1 and sigma^2.2), one for between-study (or between-DOI) heterogeneity and one for within-study (or within-DOI) heterogeneity. So, if you want to say something about the amount of heterogeneity accounted for by the moderators, then you can do the same thing that we do for a standard random/mixed-effects meta-regression model, namely compute the proportional reduction in the variance, except that we can now do this for each variance component separately:
>
> (no_mods$sigma2 - all_mods$sigma2) / no_mods$sigma2
>
> You can also compute how much the *total* amount of heterogeneity (between + within) is reduced:
>
> (sum(no_mods$sigma2) - sum(all_mods$sigma2)) / sum(no_mods$sigma2)
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> > Of max doering via R-sig-meta-analysis
> > Sent: Thursday, March 20, 2025 13:48
> > To: r-sig-meta-analysis using r-project.org
> > Cc: max doering <1maxdoering using gmail.com>
> > Subject: [R-meta] Quantify the amount of residual heterogeneity using the QE
> > value
> >
> > Dear R-sig-meta-analysis community,
> >
> > I am currently conducting a meta analysis using metafor.
> > For this, I want to compare a model with no moderators and one with
> > all moderators:
> >
> > no_mods = rma.mv(yi, vi, random = ~1|DOI/individual_level, data = data)
> > all_mods = rma.mv(yi, vi, random = ~1|DOI/individual_level, mods =
> > ~mod1 + mod2 + ..., data = data)
> >
> > I would like to answer the question, how much of the residual
> > heterogeneity in the no_mod model can be explained by adding all the
> > moderators. In the end, I want to say something like "the moderators
> > account for X% of the residual heterogeneity".
> >
> > Unfortunately, the model does not calculate a tau^2 value (at least it
> > is 0 in both models), so my thought was that I could maybe use the QE
> > value for a calculation like:
> >
> > (1 - ((all_mods $QE/all_mods $QEdf)/(no_mods $QE/no_mods $QEdf)))*100
> >
> > basically calculating the change of the QE value in percent with
> > respect to their dfs.
> >
> > In summary, my questions are:
> > 1) Can I use the QE value to quantify the amount of residual
> > heterogeneity accounted for by all moderators?
> > 2) If the answer is no or if there is a simpler solution, what options
> > are there?
> >
> > Thank you and best regards,
> > Max