[R-meta] guidance for modeling SMCC type effect size

[Reza Norouzian]

I wonder how conditions could have their own autoregressive structure?

At the study level, you could add *the same random-effect value* to
all the effects (i.e., rows) in each study. But within each study, at
the condition level, you could add *the same random-effect value* to
each set of effects (i.e., rows) that represent the same condition.

Therefore, the two (study-level and condition-level) auto-regressive
structures may lead to different estimates of correlation; one
representing the common correlation among the adjacent interval_id
levels across studies, the other representing the common correlation
among the adjacent interval_id levels across the conditions nested in
studies.

Likewise, the estimates of heterogeneity for SMCC effects at each
level of interval_id across the studies may be different from that
across the conditions nested in studies (assuming an HAR structure).

why that could be necessary?

There is nothing necessary, it's just an(other) assumption about the
structure of true effects, you can fit a model that employs such an
assumption and see if that improves the fit of the model to the data
relative to a model that doesn't utilize that assumption.

Best,
Reza




On Sun, Sep 12, 2021 at 4:43 PM Stefanou Revesz
<stefanourevesz using gmail.com> wrote:
>
> Hi Reza,
>
> Thank you so much! A quick follow-up, when you say "If in each study,
> true SMCCs at all intervals across the conditions are assumed to have
> their own auto-regressively correlated structure as well", I wonder
> how conditions could have their own autoregressive structure and why
> that could be necessary?
>
> Thanks again!
>
> On Sun, Sep 12, 2021 at 2:57 PM Reza Norouzian <rnorouzian using gmail.com> wrote:
> >
> > Dear Stefanou,
> >
> > The modeling functions like rma.mv(), rma.uni() etc. don't generally
> > depend on the type of effect size. That aside, just to make sure, did
> > you compute the change from each pre-test to each follow-up post-test,
> > or the change from each testing occasion to the following one (e.g.,
> > pre-test to post-test1, post-test1 to post-test2 ...)?
> >
> > Regardless of how you define the intervals, if your studies include a
> > control (C) and multiple treatment conditions (T1, T2,...) that occur
> > across the studies, you can create a 'condition' variable to
> > distinguish between the control and the treatments' SMCCs (yi) in each
> > study over the intervals you have defined:
> >
> > study condition  yi  interval_id id
> > 1     T1        .1   0           1
> > 1     T1        .3   1           2
> > 1     T2        .7   0           3
> > 1     T2        .2   1           4
> > 1     C         .4   0           5
> > 1     C         .5   1           6
> > 2     T2        .6   0           7
> > 2     C         .9   1           8
> >
> > In which case, a starting point might be:
> >
> > rma.mv(yi ~ condition*interval_id, V = Some_V_matrix, random = list(~
> > interval_id | study, ~ 1 | id), struct = "HAR")
> >
> > This model assumes that in each study, true SMCCs at all intervals are
> > auto-regressively correlated with each other regardless of the
> > conditions they belong to. If in each study, true SMCCs at all
> > intervals across the conditions are assumed to have their own
> > auto-regressively correlated structure as well, then, you can
> > consider:
> >
> > rma.mv(yi ~ condition*interval_id, V = Some_V_matrix, random = list(~
> > interval_id | study, ~ interval_id | interaction(study,condition), ~ 1
> > | id), struct = c("HAR","HAR") )
> >
> > In both models, when the interaction term is decomposed to its simple
> > effects for each condition, you get the average SMCC for each
> > condition (e.g., T1, T2, or C) across the intervals for your studies.
> >
> > For your second type of effect size (y_i = SMCC_T - SMCC_C, v_i =
> > v_{i_{SMCC_T}} + v_{i_{SMCC_C}}), pretty much everything I said above
> > applies. However, in this case, you're basically modeling some sort of
> > simple effect for each treatment's change vs the control's change
> > across the intervals for your studies. So, by fitting the above model
> > but using this type of effect size, you'll be asking how such simple
> > effects change over the interval you have considered.
> >
> > I believe these metrics for effect size are nowadays less commonly
> > used, partly because you can use an SMD metric, which among other
> > things doesn't require direct knowledge of pre-post correlations for
> > their computation, model them using multivariate-multilevel models,
> > and present the results in perhaps more intuitive ways to your
> > audience.
> >
> > Does that help?
> >
> > Reza
> >
> >
> >
> > Reza
> >
> >
> > On Sun, Sep 12, 2021 at 6:15 AM Stefanou Revesz
> > <stefanourevesz using gmail.com> wrote:
> > >
> > >
> > > Dear Experts,
> > >
> > > I need some guidance for modeling two types of effect sizes (for two
> > > different meta-analyses) using the rma.mv() program in metafor.
> > >
> > > First, I have computed the standardized mean change (SMCC) between
> > > each pre-test and the follow-up post-tests in each of
> > > multiple-treatment studies.
> > >
> > > Second, I have computed the difference in standardized mean changes
> > > (SMCC) between a treatment and a control group at each pre-test to
> > > post-test intervals in each of multiple-treatment studies.
> > >
> > > The reason I want to use rma.mv() is that each of my studies could
> > > produce multiple of these types of effect sizes. But I wonder what
> > > would be a starting point for modeling these two types of effect size?
> > >
> > > Any help is appreciated,
> > > Stefanou
> > >
> > > _______________________________________________
> > > R-sig-meta-analysis mailing list
> > > R-sig-meta-analysis using r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis