[R-meta] Multilevel meta-analysis with a categorical moderator | subgroup analysis using meta-regression
Viechtbauer, Wolfgang (NP)
wo||g@ng@v|echtb@uer @end|ng |rom m@@@tr|chtun|ver@|ty@n|
Thu Apr 18 18:57:05 CEST 2024
There is a good chance that I might have misremembered this, but I just double-checked -- no NAs allowed in the random part.
> -----Original Message-----
> From: Reza Norouzian <rnorouzian using gmail.com>
> Sent: Thursday, April 18, 2024 16:31
> To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Cc: Katharina Agethen <katharina.agethen using th-owl.de>; R Special Interest Group
> for Meta-Analysis <r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] Multilevel meta-analysis with a categorical moderator |
> subgroup analysis using meta-regression
>
> I would take the software developer's advice on this one;-)
>
> On Thu, Apr 18, 2024 at 9:28 AM Viechtbauer, Wolfgang (NP)
> <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> Yes, http://rma.mv() is a bit picky with missing values in variables specified
> via the 'random' argument.
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: Katharina Agethen <mailto:katharina.agethen using th-owl.de>
> > Sent: Thursday, April 18, 2024 16:22
> > To: Viechtbauer, Wolfgang (NP)
> <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>;
> > Reza Norouzian <mailto:rnorouzian using gmail.com>
> > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> > http://project.org>
> > Subject: AW: [R-meta] Multilevel meta-analysis with a categorical moderator |
> > subgroup analysis using meta-regression
> >
> > Thanks for the explanations and especially for pointing out the importance of
> > profiling, Wolfgang and Reza!
> >
> > I have one more question:
> > In my meta-analysis I have a different moderator consisting of two categories
> > (generality = global vs. contextual). For some studies I have missing values
> > (NAs) for the category.
> >
> > I have to delete those studies before specifying the single model, right?
> >
> > df.gen <- df[!http://is.na(df$generality), ]
> >
> > mod.generality <- http://rma.mv(yi = z,
> > V = vz,
> > slab = auth,
> > data = df.gen,
> > random = list(~generality | samid, ~generality |
> > esid),
> > test = "t",
> > method = "REML",
> > dfs="contain",
> > mods = ~ generality-1,
> > struct = c("DIAG", "DIAG"))
> > profile(mod.generality)
> > mod.generality.robust <- robust(mod.generality, cluster=df.gen$samid,
> > clubSandwich = TRUE)
> > summary(mod.generality.robust )
> >
> > Thanks,
> > Katharina
> >
> > -----Ursprüngliche Nachricht-----
> > Von: Viechtbauer, Wolfgang (NP)
> <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>
> > Gesendet: Thursday, April 18, 2024 12:09 PM
> > An: Reza Norouzian <mailto:rnorouzian using gmail.com>; Katharina Agethen
> > <mailto:katharina.agethen using th-owl.de>
> > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> > http://project.org>
> > Betreff: RE: [R-meta] Multilevel meta-analysis with a categorical moderator |
> > subgroup analysis using meta-regression
> >
> > I wanted to double-check that the equivalence also holds when using cluster-
> > robust variance estimation, so I ran the following example.
> >
> > ###################################
> >
> > library(metafor)
> >
> > dat <- dat.assink2016
> >
> > # standard multilevel random-effects model res <- http://rma.mv(yi, vi, random
> = ~ 1 |
> > study/id, data=dat) res
> >
> > # allow the mean effect and the between- and within-study variances to differ
> > across 'deltype'
> > res <- http://rma.mv(yi, vi, mods = ~ 0 + deltype,
> > random = list(~ deltype | study, ~ deltype | id),
> > struct="DIAG", data=dat)
> > res
> >
> > # profile the variance components
> > profile(res)
> >
> > # standard multilevel random-effects model within each deltype subset
> > res1 <- http://rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
> > subset=deltype=="covert")
> > res2 <- http://rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
> > subset=deltype=="general")
> > res3 <- http://rma.mv(yi, vi, random = ~ 1 | study/id, data=dat,
> > subset=deltype=="overt")
> >
> > # compare the estimates of the fixed effects round(coef(summary(res)), 4)
> > round(coef(summary(res1)), 4) round(coef(summary(res2)), 4)
> > round(coef(summary(res3)), 4)
> >
> > # compare the estimates of the variances round(rbind(res$tau2, res$gamma2), 4)
> > round(sapply(list(res1, res2, res3), \(x) x$sigma2), 4)
> >
> > # cluster-robust variance estimation
> > round(coef(summary(robust(res, cluster=study, clubSandwich=TRUE))), 4)
> > #round(coef(summary(robust(res1, cluster=study, clubSandwich=TRUE))), 4) #
> > cannot be run round(coef(summary(robust(res2, cluster=study,
> > clubSandwich=TRUE))), 4) round(coef(summary(robust(res3, cluster=study,
> > clubSandwich=TRUE))), 4)
> >
> > ###################################
> >
> > The profiling reveals that tau^2_1 is not identifiable -- the profile is
> > entirely flat -- because there is a single study for deltype 'covert'. Note
> that
> > when fitting model 'res1', http://rma.mv() detects this issue and sets the
> study-level
> > sigma^2 equal to 0. However, this check isn't done when fitting model 'res'.
> So
> > while the fixed effect estimate for deltype 'covert' is fine, the
> corresponding
> > SE doesn't make sense. This goes to show that profiling is important.
> >
> > The example also shows that the rest of the estimates match with the two
> > approaches. This is also true when using cluster-robust inference methods
> > (except that this method cannot be used on 'res1' since there is a single
> > cluster.
> >
> > Best,
> > Wolfgang
> >
> > > -----Original Message-----
> > > From: Reza Norouzian <mailto:rnorouzian using gmail.com>
> > > Sent: Wednesday, April 17, 2024 01:34
> > > To: Katharina Agethen <mailto:katharina.agethen using th-owl.de>
> > > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> > > http://project.org>; Viechtbauer, Wolfgang (NP)
> > > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>
> > > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
> > > moderator | subgroup analysis using meta-regression
> > >
> > > Hi Katharina,
> > >
> > > The model that I linked to in my previous response is simply
> > > equivalent to (and a shortcut for) running your "full.model" three
> > > times, each time setting the "subset=" argument in your http://rma.mv() call
> > > to one of the "pertype" categories (as Wolfgang correctly noted, make sure
> you
> > profile your variance estimates).
> > >
> > > This shortcut is also nice as it allows you to test hypotheses that
> > > involve one or more of the "pertype" categories from a single model.
> > >
> > > Please feel free to clarify what you want to achieve and we may be
> > > able to further assist you.
> > >
> > > Reza
> > >
> > > ps. It may also be useful to think about: why, in the first place, you
> > > want to conduct a subgroup analysis? I usually conduct subgroup
> > > analysis of the sort you inquired about when, among other things, (a)
> > > the categories of the moderator of interest are theoretically quite
> > > distinct, (b) I'm not willing to allow any "push-and-pull" between the
> > > categories of that moderator when some of the studies contain more
> > > than one category of the moderator in them, (c) there is a strong
> > > tradition that requires conducting this type of subgroup analysis in
> > > the relevant field of research, and (d) the subgroup shortcut model is
> > > indeed empirically (e.g., via AICc) superior to its "push-and-pull"
> > alternative (the "mod.pertype" model in your example).
> > >
> > > On Tue, Apr 16, 2024 at 11:12 AM Viechtbauer, Wolfgang (NP)
> > > <mailto:mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > > There should be three tau^2 values and three gamma^2 values. The
> > > former are the between-sample variances and the later are the within-sample
> > variances.
> > >
> > > I would strongly suggest to run profile() on this model to check that
> > > all variance components are identifiable.
> > >
> > > https://wviechtb.github.io/metafor/reference/profile.rma.html
> > >
> > > Best,
> > > Wolfgang
> > >
> > > > -----Original Message-----
> > > > From: R-sig-meta-analysis
> > > > <mailto:mailto:r-sig-meta-analysis-bounces using r-project.org>
> > > On Behalf
> > > > Of Katharina Agethen via R-sig-meta-analysis
> > > > Sent: Tuesday, April 16, 2024 17:04
> > > > To: Reza Norouzian <mailto:mailto:rnorouzian using gmail.com>; R Special
> Interest
> > > > Group for
> > > Meta-
> > > > Analysis <mailto:mailto:r-sig-meta-analysis using r-project.org>
> > > > Cc: Katharina Agethen <mailto:mailto:katharina.agethen using th-owl.de>
> > > > Subject: Re: [R-meta] Multilevel meta-analysis with a categorical
> > > > moderator | subgroup analysis using meta-regression
> > > >
> > > > Hi Reza,
> > > >
> > > > Thank you so much for your prompt reply. That is really helpful to me.
> > > >
> > > > After adjusting my code, I now obtain three tau^2 values based on my
> > > > three subgroups. Does it still make sense to compute the variance
> > > > components for the different levels (between/within sample) so that
> > > > I may report the variance components of the "overall" predictor
> > > > "pertype"? If needed, how would I
> > > compute
> > > > tau^2(3) (between samples) and tau^2(2) (within samples) for the
> > > > moderation analysis?
> > > >
> > > > Best,
> > > > Katharina
More information about the R-sig-meta-analysis
mailing list