[R-meta] guidance for modeling SMCC type effect size

[Stefanou Revesz]

Very interesting! Thanks a lot! Just a very final question, what does
the ~ 1 | id part do or assume?

On Sun, Sep 12, 2021 at 6:13 PM Reza Norouzian <rnorouzian using gmail.com> wrote:
>
> I wonder how conditions could have their own autoregressive structure?
>
> At the study level, you could add *the same random-effect value* to
> all the effects (i.e., rows) in each study. But within each study, at
> the condition level, you could add *the same random-effect value* to
> each set of effects (i.e., rows) that represent the same condition.
>
> Therefore, the two (study-level and condition-level) auto-regressive
> structures may lead to different estimates of correlation; one
> representing the common correlation among the adjacent interval_id
> levels across studies, the other representing the common correlation
> among the adjacent interval_id levels across the conditions nested in
> studies.
>
> Likewise, the estimates of heterogeneity for SMCC effects at each
> level of interval_id across the studies may be different from that
> across the conditions nested in studies (assuming an HAR structure).
>
> why that could be necessary?
>
> There is nothing necessary, it's just an(other) assumption about the
> structure of true effects, you can fit a model that employs such an
> assumption and see if that improves the fit of the model to the data
> relative to a model that doesn't utilize that assumption.
>
> Best,
> Reza
>
>
>
>
> On Sun, Sep 12, 2021 at 4:43 PM Stefanou Revesz
> <stefanourevesz using gmail.com> wrote:
> >
> > Hi Reza,
> >
> > Thank you so much! A quick follow-up, when you say "If in each study,
> > true SMCCs at all intervals across the conditions are assumed to have
> > their own auto-regressively correlated structure as well", I wonder
> > how conditions could have their own autoregressive structure and why
> > that could be necessary?
> >
> > Thanks again!
> >
> > On Sun, Sep 12, 2021 at 2:57 PM Reza Norouzian <rnorouzian using gmail.com> wrote:
> > >
> > > Dear Stefanou,
> > >
> > > The modeling functions like rma.mv(), rma.uni() etc. don't generally
> > > depend on the type of effect size. That aside, just to make sure, did
> > > you compute the change from each pre-test to each follow-up post-test,
> > > or the change from each testing occasion to the following one (e.g.,
> > > pre-test to post-test1, post-test1 to post-test2 ...)?
> > >
> > > Regardless of how you define the intervals, if your studies include a
> > > control (C) and multiple treatment conditions (T1, T2,...) that occur
> > > across the studies, you can create a 'condition' variable to
> > > distinguish between the control and the treatments' SMCCs (yi) in each
> > > study over the intervals you have defined:
> > >
> > > study condition  yi  interval_id id
> > > 1     T1        .1   0           1
> > > 1     T1        .3   1           2
> > > 1     T2        .7   0           3
> > > 1     T2        .2   1           4
> > > 1     C         .4   0           5
> > > 1     C         .5   1           6
> > > 2     T2        .6   0           7
> > > 2     C         .9   1           8
> > >
> > > In which case, a starting point might be:
> > >
> > > rma.mv(yi ~ condition*interval_id, V = Some_V_matrix, random = list(~
> > > interval_id | study, ~ 1 | id), struct = "HAR")
> > >
> > > This model assumes that in each study, true SMCCs at all intervals are
> > > auto-regressively correlated with each other regardless of the
> > > conditions they belong to. If in each study, true SMCCs at all
> > > intervals across the conditions are assumed to have their own
> > > auto-regressively correlated structure as well, then, you can
> > > consider:
> > >
> > > rma.mv(yi ~ condition*interval_id, V = Some_V_matrix, random = list(~
> > > interval_id | study, ~ interval_id | interaction(study,condition), ~ 1
> > > | id), struct = c("HAR","HAR") )
> > >
> > > In both models, when the interaction term is decomposed to its simple
> > > effects for each condition, you get the average SMCC for each
> > > condition (e.g., T1, T2, or C) across the intervals for your studies.
> > >
> > > For your second type of effect size (y_i = SMCC_T - SMCC_C, v_i =
> > > v_{i_{SMCC_T}} + v_{i_{SMCC_C}}), pretty much everything I said above
> > > applies. However, in this case, you're basically modeling some sort of
> > > simple effect for each treatment's change vs the control's change
> > > across the intervals for your studies. So, by fitting the above model
> > > but using this type of effect size, you'll be asking how such simple
> > > effects change over the interval you have considered.
> > >
> > > I believe these metrics for effect size are nowadays less commonly
> > > used, partly because you can use an SMD metric, which among other
> > > things doesn't require direct knowledge of pre-post correlations for
> > > their computation, model them using multivariate-multilevel models,
> > > and present the results in perhaps more intuitive ways to your
> > > audience.
> > >
> > > Does that help?
> > >
> > > Reza
> > >
> > >
> > >
> > > Reza
> > >
> > >
> > > On Sun, Sep 12, 2021 at 6:15 AM Stefanou Revesz
> > > <stefanourevesz using gmail.com> wrote:
> > > >
> > > >
> > > > Dear Experts,
> > > >
> > > > I need some guidance for modeling two types of effect sizes (for two
> > > > different meta-analyses) using the rma.mv() program in metafor.
> > > >
> > > > First, I have computed the standardized mean change (SMCC) between
> > > > each pre-test and the follow-up post-tests in each of
> > > > multiple-treatment studies.
> > > >
> > > > Second, I have computed the difference in standardized mean changes
> > > > (SMCC) between a treatment and a control group at each pre-test to
> > > > post-test intervals in each of multiple-treatment studies.
> > > >
> > > > The reason I want to use rma.mv() is that each of my studies could
> > > > produce multiple of these types of effect sizes. But I wonder what
> > > > would be a starting point for modeling these two types of effect size?
> > > >
> > > > Any help is appreciated,
> > > > Stefanou
> > > >
> > > > _______________________________________________
> > > > R-sig-meta-analysis mailing list
> > > > R-sig-meta-analysis using r-project.org
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis