[R-meta] Heterogeneity of Multilevel Moderator Analysis

[Reza Norouzian]

Dear Tori,

Purely theoretically (not knowing your data, goals etc.) the
following allows you to differentiate and estimate the random variation in
the true effects across the categories of DISSERTATION at the study and the
effect size levels:

t1 = rma.mv(yi = yi,
             V = vi,
       mods = ~DISSERTATION-1,
         slab = STUDY_ID,
        data = Relationship,
       random = list(~DISSERTATION | STUDY_ID,
~DISSERTATION|EFFECT_SIZE_ID),
        test = "t",
       struct = c("DIAG","DIAG"))

It's nice that metafor allows doing this so complactly using a single model
(but at no more than two levels). For one thing, otherwise, you would have
needed to fit as many models as there are categories in your categorical
variable.

For example, if DISSERTATION, has two categories (yes and no), then the
above compact model will be equivalent to running the following two models:

t2 = rma.mv(yi, vi,
  random = ~1|STUDY_ID/EFFECT_SIZE_ID,
       data = Relationship,
   subset = DISSERTATION == "yes",
       test = "t")

t3 = rma.mv(yi, vi,
  random = ~1|STUDY_ID/EFFECT_SIZE_ID,
       data =  Relationship,
   subset = DISSERTATION == "no",
        test = "t")

For another thing at least in metafor (try the clubSandwich package as this
may or may not be true there), using a single compact model would use the
entire data for the purpose of inferential conclusions which if you use a
t-distribution (which you currently do) for inferential purposes can affect
the p-values and the CIs that you get for your fixed effects from the
compact model versus the separate individual models.

Kind regards,
Reza


On Thu, Jun 16, 2022 at 10:31 AM Tori Peña <Tori.Pena using stonybrook.edu> wrote:

> Hi folks,
>
> Thanks for all your help!
>
> I was curious if there is a way to get the variance composition of a
> moderator analysis across the different categorical levels?
>
> For example, I have a three-level meta-analysis with publication status as
> the predictor below. When I enter the model into var.comp, I get the
> overall I^2 across the different levels, but not between published versus
> unpublished studies.
>
> m_1_pub <- rma.mv(yi = yi,
>                      V = vi,
>                      mods = ~ DISSERTATION,
>                      slab = STUDY_ID,
>                      data = Relationship,
>                      random = ~ 1 | STUDY_ID/EFFECT_SIZE_ID,
>                      test = "t",
>                      method = "REML")
>
> i2_m_1_pub <- var.comp(m_1_pub)
> summary(i2_m_1_pub)
> print(m_1_pub$tau2)
>
> Thanks again,
> Tori
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]