[R-meta] To include or not include time in rma.mv

[Yuhang Hu]

Dear James,

Thank you very much for the clarification.

I'm assuming that you guys excluded the pretreatment effects mainly because
your study pool consisted only of true experiments (right?).

I might be wrong, but random assignment in single experiments could still
suffer from bad luck and can equalize the groups only on average given
innumerable replications.

So, do you advise including a "time" interaction overall?

Thank you so much for your time.

Yuhang

On Tue, Sep 6, 2022 at 8:04 AM James Pustejovsky <jepusto using gmail.com> wrote:

> Hi Yuhang,
>
> As one of the authors of the second study you mentioned, I think you may
> be misinterpreting our analysis. My understanding is that the data that we
> used to develop our examples did not include standardized mean differences
> prior to treatment. All of the included effect sizes were for
> post-treatment effects (i.e., `postwks >= 0`).
>
> I can't say for certain, but I suspect that the same thing holds for the
> Williams et al. meta-analysis (the first study you mentioned). If you look
> at their Table 2, the "Outcome timing" variable ranges from "midstream
> during intervention" to "follow-up post-test," so there don't appear to be
> any pre-treatment effect sizes included in the analysis.
>
> If the data did include pre-test effect sizes, then I fully agree that m1
> would blur the interpretation of the treatment effects.
>
> James
>
> On Mon, Sep 5, 2022 at 7:36 PM Yuhang Hu <yh342 using nau.edu> wrote:
>
>> Hello All,
>>
>> I'm considering two candidate fixed-effects structures (cat_mod
>> = categorical moderator) to meta-analyze a group of longitudinal
>> studies (pre-test and follow-ups):
>>
>> m1 = rma.mv(yi ~ 0 + cat_mod + covariates, random = ~ 1 | study/effect)
>> m2 = rma.mv(yi ~ 0 + cat_mod*time + covariates, random = ~ 1 |
>> study/effect)
>>
>> In several authoritative meta-analyses of longitudinal studies such as
>> ***https://doi.org/10.1080/19345747.2021.2009072*** and
>> ***https://doi.org/10.1007/s11121-021-01246-3***, I noticed the authors
>> used "m1" over "m2".
>>
>> But I was wondering, wouldn't, ignoring "time" (as in "m1") in
>> the fixed-effects structure mix up the pre-test and follow-up effect
>> sizes together which can blur our understanding of the treatment effect?
>>
>> Your guidance is highly appreciated,
>>
>> Yuhang
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
>> To manage your subscription to this mailing list, go to:
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>

-- 
Yuhang Hu (She/Her/Hers)
Ph.D. Student in Applied Linguistics
Department of English
Northern Arizona University

	[[alternative HTML version deleted]]