[R-meta] Quantify the amount of residual heterogeneity using the QE value

[Viechtbauer, Wolfgang (NP)]

Don't tempt me ...

> -----Original Message-----
> From: max doering <1maxdoering using gmail.com>
> Sent: Thursday, March 20, 2025 15:10
> To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Subject: Re: [R-meta] Quantify the amount of residual heterogeneity using the QE
> value
>
> Dear Wolfgang,
> thank you so much!
> You should really consider implementing a parameter called chicken^2...
> Best,
> Max
>
> Am Do., 20. März 2025 um 14:24 Uhr schrieb Viechtbauer, Wolfgang (NP)
> <wolfgang.viechtbauer using maastrichtuniversity.nl>:
> >
> > Dear Max,
> >
> > Whether we call a variance component tau^2 or sigma^2 or chicken^2 really
> doesn't matter. When I fit a standard random-effects model using rma.mv() as
> shown here:
> >
> > https://www.metafor-project.org/doku.php/analyses:konstantopoulos2011
> >
> > or here:
> >
> > https://www.metafor-project.org/doku.php/analyses:crede2010
> >
> > then the variance component (often denoted tau^2 in the literature, but not
> always so) which reflects the amount of heterogeneity in the true effect sizes
> is denoted sigma^2 in the output. It's still the same parameter.
> >
> > In the multilevel model, there are two variance components (denoted sigma^2.1
> and sigma^2.2), one for between-study (or between-DOI) heterogeneity and one for
> within-study (or within-DOI) heterogeneity. So, if you want to say something
> about the amount of heterogeneity accounted for by the moderators, then you can
> do the same thing that we do for a standard random/mixed-effects meta-regression
> model, namely compute the proportional reduction in the variance, except that we
> can now do this for each variance component separately:
> >
> > (no_mods$sigma2 - all_mods$sigma2) / no_mods$sigma2
> >
> > You can also compute how much the *total* amount of heterogeneity (between +
> within) is reduced:
> >
> > (sum(no_mods$sigma2) - sum(all_mods$sigma2)) / sum(no_mods$sigma2)
> >
> > Best,
> > Wolfgang
> >
> > > -----Original Message-----
> > > From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On
> Behalf
> > > Of max doering via R-sig-meta-analysis
> > > Sent: Thursday, March 20, 2025 13:48
> > > To: r-sig-meta-analysis using r-project.org
> > > Cc: max doering <1maxdoering using gmail.com>
> > > Subject: [R-meta] Quantify the amount of residual heterogeneity using the QE
> > > value
> > >
> > > Dear R-sig-meta-analysis community,
> > >
> > > I am currently conducting a meta analysis using metafor.
> > > For this, I want to compare a model with no moderators and one with
> > > all moderators:
> > >
> > > no_mods = rma.mv(yi, vi, random = ~1|DOI/individual_level, data = data)
> > > all_mods = rma.mv(yi, vi, random = ~1|DOI/individual_level, mods =
> > > ~mod1 + mod2 + ..., data = data)
> > >
> > > I would like to answer the question, how much of the residual
> > > heterogeneity in the no_mod model can be explained by adding all the
> > > moderators. In the end, I want to say something like "the moderators
> > > account for X% of the residual heterogeneity".
> > >
> > > Unfortunately, the model does not calculate a tau^2 value (at least it
> > > is 0 in both models), so my thought was that I could maybe use the QE
> > > value for a calculation like:
> > >
> > > (1 - ((all_mods $QE/all_mods $QEdf)/(no_mods $QE/no_mods $QEdf)))*100
> > >
> > > basically calculating the change of the QE value in percent with
> > > respect to their dfs.
> > >
> > > In summary, my questions are:
> > > 1) Can I use the QE value to quantify the amount of residual
> > > heterogeneity accounted for by all moderators?
> > > 2) If the answer is no or if there is a simpler solution, what options
> > > are there?
> > >
> > > Thank you and best regards,
> > > Max