[R-meta] Multilevel meta-analysis with a categorical moderator | subgroup analysis using meta-regression

[Viechtbauer, Wolfgang (NP)]

Yes, rma.mv() is a bit picky with missing values in variables specified via the 'random' argument.

Best,
Wolfgang

> -----Original Message-----
> From: Katharina Agethen <katharina.agethen using th-owl.de>
> Sent: Thursday, April 18, 2024 16:22
> To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>;
> Reza Norouzian <rnorouzian using gmail.com>
> Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Subject: AW: [R-meta] Multilevel meta-analysis with a categorical moderator |
> subgroup analysis using meta-regression
>
> Thanks for the explanations and especially for pointing out the importance of
> profiling, Wolfgang and Reza!
>
> I have one more question:
> In my meta-analysis I have a different moderator consisting of two categories
> (generality = global vs. contextual). For some studies I have missing values
> (NAs) for the category.
>
> I have to delete those studies before specifying the single model, right?
>
> df.gen <- df[!is.na(df$generality), ]
>
> mod.generality <-  rma.mv(yi = z,
>                           V = vz,
>                           slab = auth,
>                           data = df.gen,
>                           random = list(~generality | samid, ~generality |
> esid),
>                           test = "t",
>                           method = "REML",
>                           dfs="contain",
>                           mods = ~ generality-1,
>                           struct = c("DIAG", "DIAG"))
> profile(mod.generality)
> mod.generality.robust <- robust(mod.generality, cluster=df.gen$samid,
> clubSandwich = TRUE)
> summary(mod.generality.robust )
>
> Thanks,
> Katharina
>
> -----Ursprüngliche Nachricht-----
> Von: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Gesendet: Thursday, April 18, 2024 12:09 PM
> An: Reza Norouzian <rnorouzian using gmail.com>; Katharina Agethen
> <katharina.agethen using th-owl.de>
> Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Betreff: RE: [R-meta] Multilevel meta-analysis with a categorical moderator |
> subgroup analysis using meta-regression
>
> I wanted to double-check that the equivalence also holds when using cluster-
> robust variance estimation, so I ran the following example.
>
> ###################################
>
> library(metafor)
>
> dat <- dat.assink2016
>
> # standard multilevel random-effects model res <- rma.mv(yi, vi, random = ~ 1 |
> study/id, data=dat) res
>
> # allow the mean effect and the between- and within-study variances to differ
> across 'deltype'
> res <- rma.mv(yi, vi, mods = ~ 0 + deltype,
>               random = list(~ deltype | study, ~ deltype | id),
>               struct="DIAG", data=dat)
> res
>
> # profile the variance components
> profile(res)
>
> # standard multilevel random-effects model within each deltype subset
> res1 <- rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
> subset=deltype=="covert")
> res2 <- rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
> subset=deltype=="general")
> res3 <- rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
> subset=deltype=="overt")
>
> # compare the estimates of the fixed effects round(coef(summary(res)), 4)
> round(coef(summary(res1)), 4) round(coef(summary(res2)), 4)
> round(coef(summary(res3)), 4)
>
> # compare the estimates of the variances round(rbind(res$tau2, res$gamma2), 4)
> round(sapply(list(res1, res2, res3), \(x) x$sigma2), 4)
>
> # cluster-robust variance estimation
> round(coef(summary(robust(res, cluster=study, clubSandwich=TRUE))), 4)
> #round(coef(summary(robust(res1, cluster=study, clubSandwich=TRUE))), 4) #
> cannot be run round(coef(summary(robust(res2, cluster=study,
> clubSandwich=TRUE))), 4) round(coef(summary(robust(res3, cluster=study,
> clubSandwich=TRUE))), 4)
>
> ###################################
>
> The profiling reveals that tau^2_1 is not identifiable -- the profile is
> entirely flat -- because there is a single study for deltype 'covert'. Note that
> when fitting model 'res1', rma.mv() detects this issue and sets the study-level
> sigma^2 equal to 0. However, this check isn't done when fitting model 'res'. So
> while the fixed effect estimate for deltype 'covert' is fine, the corresponding
> SE doesn't make sense. This goes to show that profiling is important.
>
> The example also shows that the rest of the estimates match with the two
> approaches. This is also true when using cluster-robust inference methods
> (except that this method cannot be used on 'res1' since there is a single
> cluster.
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: Reza Norouzian <rnorouzian using gmail.com>
> > Sent: Wednesday, April 17, 2024 01:34
> > To: Katharina Agethen <katharina.agethen using th-owl.de>
> > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> > project.org>; Viechtbauer, Wolfgang (NP)
> > <wolfgang.viechtbauer using maastrichtuniversity.nl>
> > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
> > moderator | subgroup analysis using meta-regression
> >
> > Hi Katharina,
> >
> > The model that I linked to in my previous response is simply
> > equivalent to (and a shortcut for) running your "full.model" three
> > times, each time setting the "subset=" argument in your rma.mv() call
> > to one of the "pertype" categories (as Wolfgang correctly noted, make sure you
> profile your variance estimates).
> >
> > This shortcut is also nice as it allows you to test hypotheses that
> > involve one or more of the "pertype" categories from a single model.
> >
> > Please feel free to clarify what you want to achieve and we may be
> > able to further assist you.
> >
> > Reza
> >
> > ps. It may also be useful to think about: why, in the first place, you
> > want to conduct a subgroup analysis? I usually conduct subgroup
> > analysis of the sort you inquired about when, among other things, (a)
> > the categories of the moderator of interest are theoretically quite
> > distinct, (b) I'm not willing to allow any "push-and-pull" between the
> > categories of that moderator when some of the studies contain more
> > than one category of the moderator in them, (c) there is a strong
> > tradition that requires conducting this type of subgroup analysis in
> > the relevant field of research, and (d) the subgroup shortcut model is
> > indeed empirically (e.g., via AICc) superior to its "push-and-pull"
> alternative (the "mod.pertype" model in your example).
> >
> > On Tue, Apr 16, 2024 at 11:12 AM Viechtbauer, Wolfgang (NP)
> > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > There should be three tau^2 values and three gamma^2 values. The
> > former are the between-sample variances and the later are the within-sample
> variances.
> >
> > I would strongly suggest to run profile() on this model to check that
> > all variance components are identifiable.
> >
> > https://wviechtb.github.io/metafor/reference/profile.rma.html
> >
> > Best,
> > Wolfgang
> >
> > > -----Original Message-----
> > > From: R-sig-meta-analysis
> > > <mailto:r-sig-meta-analysis-bounces using r-project.org>
> > On Behalf
> > > Of Katharina Agethen via R-sig-meta-analysis
> > > Sent: Tuesday, April 16, 2024 17:04
> > > To: Reza Norouzian <mailto:rnorouzian using gmail.com>; R Special Interest
> > > Group for
> > Meta-
> > > Analysis <mailto:r-sig-meta-analysis using r-project.org>
> > > Cc: Katharina Agethen <mailto:katharina.agethen using th-owl.de>
> > > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
> > > moderator | subgroup analysis using meta-regression
> > >
> > > Hi Reza,
> > >
> > > Thank you so much for your prompt reply. That is really helpful to me.
> > >
> > > After adjusting my code, I now obtain three tau^2 values based on my
> > > three subgroups. Does it still make sense to compute the variance
> > > components for the different levels (between/within sample) so that
> > > I may report the variance components of the "overall" predictor
> > > "pertype"? If needed, how would I
> > compute
> > > tau^2(3) (between samples) and tau^2(2) (within samples) for the
> > > moderation analysis?
> > >
> > > Best,
> > > Katharina