[R-meta] Multilevel meta-analysis with a categorical moderator | subgroup analysis using meta-regression

Reza Norouzian rnorouz|@n @end|ng |rom gm@||@com
Thu Apr 18 22:26:55 CEST 2024 

Forgive the typo in dat2:  dat2 <- subset(dat1, !is.na(deltype))


On Thu, Apr 18, 2024 at 2:05 PM Reza Norouzian <rnorouzian using gmail.com> wrote:

> Thanks, Wolfgang. If NAs are to be removed for the single subgroup model
> to work, then, I would probably prefer to use the "res123" models than to
> remove the NAs to get a single subgroup model;-)
>
> dat <- dat.assink2016
> dat1 <- transform(dat, deltype = ifelse(year %in% c(-7.5,-11.5), NA,
> deltype))
> dat2 <- subset(dat, !is.na(deltype))
>
> res <- rma.mv(yi, vi, mods = ~ 0 + deltype,
>               random = list(~ deltype | study, ~ deltype | id),
>               struct="DIAG", data=dat2)
>
> res123 <- lapply(na.omit(unique(dat1$deltype)), \(i) rma.mv(yi, vi,
> random=~1|study/id, data=dat1, subset=deltype==i))
>
>
>
> On Thu, Apr 18, 2024 at 11:57 AM Viechtbauer, Wolfgang (NP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>
>> There is a good chance that I might have misremembered this, but I just
>> double-checked -- no NAs allowed in the random part.
>>
>> > -----Original Message-----
>> > From: Reza Norouzian <rnorouzian using gmail.com>
>> > Sent: Thursday, April 18, 2024 16:31
>> > To: Viechtbauer, Wolfgang (NP) <
>> wolfgang.viechtbauer using maastrichtuniversity.nl>
>> > Cc: Katharina Agethen <katharina.agethen using th-owl.de>; R Special
>> Interest Group
>> > for Meta-Analysis <r-sig-meta-analysis using r-project.org>
>> > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
>> moderator |
>> > subgroup analysis using meta-regression
>> >
>> > I would take the software developer's advice on this one;-)
>> >
>> > On Thu, Apr 18, 2024 at 9:28 AM Viechtbauer, Wolfgang (NP)
>> > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> > Yes, http://rma.mv() is a bit picky with missing values in variables
>> specified
>> > via the 'random' argument.
>> >
>> > Best,
>> > Wolfgang
>> >
>> > > -----Original Message-----
>> > > From: Katharina Agethen <mailto:katharina.agethen using th-owl.de>
>> > > Sent: Thursday, April 18, 2024 16:22
>> > > To: Viechtbauer, Wolfgang (NP)
>> > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>;
>> > > Reza Norouzian <mailto:rnorouzian using gmail.com>
>> > > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>> > > http://project.org>
>> > > Subject: AW: [R-meta] Multilevel meta-analysis with a categorical
>> moderator |
>> > > subgroup analysis using meta-regression
>> > >
>> > > Thanks for the explanations and especially for pointing out the
>> importance of
>> > > profiling, Wolfgang and Reza!
>> > >
>> > > I have one more question:
>> > > In my meta-analysis I have a different moderator consisting of two
>> categories
>> > > (generality = global vs. contextual). For some studies I have missing
>> values
>> > > (NAs) for the category.
>> > >
>> > > I have to delete those studies before specifying the single model,
>> right?
>> > >
>> > > df.gen <- df[!http://is.na(df$generality), ]
>> > >
>> > > mod.generality <-  http://rma.mv(yi = z,
>> > >                           V = vz,
>> > >                           slab = auth,
>> > >                           data = df.gen,
>> > >                           random = list(~generality | samid,
>> ~generality |
>> > > esid),
>> > >                           test = "t",
>> > >                           method = "REML",
>> > >                           dfs="contain",
>> > >                           mods = ~ generality-1,
>> > >                           struct = c("DIAG", "DIAG"))
>> > > profile(mod.generality)
>> > > mod.generality.robust <- robust(mod.generality, cluster=df.gen$samid,
>> > > clubSandwich = TRUE)
>> > > summary(mod.generality.robust )
>> > >
>> > > Thanks,
>> > > Katharina
>> > >
>> > > -----Ursprüngliche Nachricht-----
>> > > Von: Viechtbauer, Wolfgang (NP)
>> > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>
>> > > Gesendet: Thursday, April 18, 2024 12:09 PM
>> > > An: Reza Norouzian <mailto:rnorouzian using gmail.com>; Katharina Agethen
>> > > <mailto:katharina.agethen using th-owl.de>
>> > > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>> > > http://project.org>
>> > > Betreff: RE: [R-meta] Multilevel meta-analysis with a categorical
>> moderator |
>> > > subgroup analysis using meta-regression
>> > >
>> > > I wanted to double-check that the equivalence also holds when using
>> cluster-
>> > > robust variance estimation, so I ran the following example.
>> > >
>> > > ###################################
>> > >
>> > > library(metafor)
>> > >
>> > > dat <- dat.assink2016
>> > >
>> > > # standard multilevel random-effects model res <- http://rma.mv(yi,
>> vi, random
>> > = ~ 1 |
>> > > study/id, data=dat) res
>> > >
>> > > # allow the mean effect and the between- and within-study variances
>> to differ
>> > > across 'deltype'
>> > > res <- http://rma.mv(yi, vi, mods = ~ 0 + deltype,
>> > >               random = list(~ deltype | study, ~ deltype | id),
>> > >               struct="DIAG", data=dat)
>> > > res
>> > >
>> > > # profile the variance components
>> > > profile(res)
>> > >
>> > > # standard multilevel random-effects model within each deltype subset
>> > > res1 <- http://rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
>> > > subset=deltype=="covert")
>> > > res2 <- http://rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
>> > > subset=deltype=="general")
>> > > res3 <- http://rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
>> > > subset=deltype=="overt")
>> > >
>> > > # compare the estimates of the fixed effects
>> round(coef(summary(res)), 4)
>> > > round(coef(summary(res1)), 4) round(coef(summary(res2)), 4)
>> > > round(coef(summary(res3)), 4)
>> > >
>> > > # compare the estimates of the variances round(rbind(res$tau2,
>> res$gamma2), 4)
>> > > round(sapply(list(res1, res2, res3), \(x) x$sigma2), 4)
>> > >
>> > > # cluster-robust variance estimation
>> > > round(coef(summary(robust(res, cluster=study, clubSandwich=TRUE))), 4)
>> > > #round(coef(summary(robust(res1, cluster=study, clubSandwich=TRUE))),
>> 4) #
>> > > cannot be run round(coef(summary(robust(res2, cluster=study,
>> > > clubSandwich=TRUE))), 4) round(coef(summary(robust(res3,
>> cluster=study,
>> > > clubSandwich=TRUE))), 4)
>> > >
>> > > ###################################
>> > >
>> > > The profiling reveals that tau^2_1 is not identifiable -- the profile
>> is
>> > > entirely flat -- because there is a single study for deltype
>> 'covert'. Note
>> > that
>> > > when fitting model 'res1', http://rma.mv() detects this issue and
>> sets the
>> > study-level
>> > > sigma^2 equal to 0. However, this check isn't done when fitting model
>> 'res'.
>> > So
>> > > while the fixed effect estimate for deltype 'covert' is fine, the
>> > corresponding
>> > > SE doesn't make sense. This goes to show that profiling is important.
>> > >
>> > > The example also shows that the rest of the estimates match with the
>> two
>> > > approaches. This is also true when using cluster-robust inference
>> methods
>> > > (except that this method cannot be used on 'res1' since there is a
>> single
>> > > cluster.
>> > >
>> > > Best,
>> > > Wolfgang
>> > >
>> > > > -----Original Message-----
>> > > > From: Reza Norouzian <mailto:rnorouzian using gmail.com>
>> > > > Sent: Wednesday, April 17, 2024 01:34
>> > > > To: Katharina Agethen <mailto:katharina.agethen using th-owl.de>
>> > > > Cc: R Special Interest Group for Meta-Analysis
>> <r-sig-meta-analysis using r-
>> > > > http://project.org>; Viechtbauer, Wolfgang (NP)
>> > > > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>
>> > > > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
>> > > > moderator | subgroup analysis using meta-regression
>> > > >
>> > > > Hi Katharina,
>> > > >
>> > > > The model that I linked to in my previous response is simply
>> > > > equivalent to (and a shortcut for) running your "full.model" three
>> > > > times, each time setting the "subset=" argument in your
>> http://rma.mv() call
>> > > > to one of the "pertype" categories (as Wolfgang correctly noted,
>> make sure
>> > you
>> > > profile your variance estimates).
>> > > >
>> > > > This shortcut is also nice as it allows you to test hypotheses that
>> > > > involve one or more of the "pertype" categories from a single model.
>> > > >
>> > > > Please feel free to clarify what you want to achieve and we may be
>> > > > able to further assist you.
>> > > >
>> > > > Reza
>> > > >
>> > > > ps. It may also be useful to think about: why, in the first place,
>> you
>> > > > want to conduct a subgroup analysis? I usually conduct subgroup
>> > > > analysis of the sort you inquired about when, among other things,
>> (a)
>> > > > the categories of the moderator of interest are theoretically quite
>> > > > distinct, (b) I'm not willing to allow any "push-and-pull" between
>> the
>> > > > categories of that moderator when some of the studies contain more
>> > > > than one category of the moderator in them, (c) there is a strong
>> > > > tradition that requires conducting this type of subgroup analysis in
>> > > > the relevant field of research, and (d) the subgroup shortcut model
>> is
>> > > > indeed empirically (e.g., via AICc) superior to its "push-and-pull"
>> > > alternative (the "mod.pertype" model in your example).
>> > > >
>> > > > On Tue, Apr 16, 2024 at 11:12 AM Viechtbauer, Wolfgang (NP)
>> > > > <mailto:mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> > > > There should be three tau^2 values and three gamma^2 values. The
>> > > > former are the between-sample variances and the later are the
>> within-sample
>> > > variances.
>> > > >
>> > > > I would strongly suggest to run profile() on this model to check
>> that
>> > > > all variance components are identifiable.
>> > > >
>> > > > https://wviechtb.github.io/metafor/reference/profile.rma.html
>> > > >
>> > > > Best,
>> > > > Wolfgang
>> > > >
>> > > > > -----Original Message-----
>> > > > > From: R-sig-meta-analysis
>> > > > > <mailto:mailto:r-sig-meta-analysis-bounces using r-project.org>
>> > > > On Behalf
>> > > > > Of Katharina Agethen via R-sig-meta-analysis
>> > > > > Sent: Tuesday, April 16, 2024 17:04
>> > > > > To: Reza Norouzian <mailto:mailto:rnorouzian using gmail.com>; R
>> Special
>> > Interest
>> > > > > Group for
>> > > > Meta-
>> > > > > Analysis <mailto:mailto:r-sig-meta-analysis using r-project.org>
>> > > > > Cc: Katharina Agethen <mailto:mailto:katharina.agethen using th-owl.de>
>> > > > > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
>> > > > > moderator | subgroup analysis using meta-regression
>> > > > >
>> > > > > Hi Reza,
>> > > > >
>> > > > > Thank you so much for your prompt reply. That is really helpful
>> to me.
>> > > > >
>> > > > > After adjusting my code, I now obtain three tau^2 values based on
>> my
>> > > > > three subgroups. Does it still make sense to compute the variance
>> > > > > components for the different levels (between/within sample) so
>> that
>> > > > > I may report the variance components of the "overall" predictor
>> > > > > "pertype"? If needed, how would I
>> > > > compute
>> > > > > tau^2(3) (between samples) and tau^2(2) (within samples) for the
>> > > > > moderation analysis?
>> > > > >
>> > > > > Best,
>> > > > > Katharina
>>
>

	[[alternative HTML version deleted]]

More information about the R-sig-meta-analysis mailing list