[R-meta] Estimating "overall effect" in meta-regression

[Viechtbauer, Wolfgang (SP)]

For such a pre-post-control-group design design, one could compute two standardized mean differences (one for time 1 and one for time 2), or two standardized mean changes (one for the treatment and one for the control group), or one could compute the difference between two such estimates.

Based on the schema below, it looks like you are computing two standardized mean differences (one for time 1 and one for time 2). Note that the two standardized mean differences are not independent, since they are computed based on the same subjects. This aside, then one definitely should include time as a moderator, as it would make little sense to synthesize estimates from before and after the treatment into a single pooled effect.

Best,
Wolfgang

>-----Original Message-----
>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>Sent: Monday, 10 January, 2022 13:38
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: Estimating "overall effect" in meta-regression
>
>Dear Wolfgang,
>
>The studies follow a pre-post-control design. The effect size measure used is
>standardized mean difference (SMD).
>
>I hope this clarifies my question.
>Luke
>
>sudy time  yi
>1         0
>1         1
>2         0
>2         1
>
>On Mon, Jan 10, 2022, 5:43 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>Dear Luke,
>
>I don't understand the question, in part because it is not clear to me what kind
>of design the studies have and what kind of effect size measure you are using.
>Could you clarify this?
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>>Sent: Sunday, 09 January, 2022 7:53
>>To: R meta
>>Cc: Viechtbauer, Wolfgang (SP)
>>Subject: Estimating "overall effect" in meta-regression
>>
>>Dear R-meta Community,
>>
>>I'm meta-analyzing a group of pre-post studies. My first RQ is: what
>>is the "overall effect" of policy X on a dependent variable.
>>
>>I know that I can fit an intercept-only model to answer this RQ.
>>
>>But an intercept-only model estimates the average effect size across
>>BOTH pre-tests (before policy X implementation) and post-tests (after
>>policy X implementation), while pre-test effect sizes don't contain
>>any policy X effect.
>>
>>Given that, is it appropriate to use an intercept-only model to answer
>>this RQ, or I actually need to use a time indicator as a moderator to
>>separate the pre- from post-test effect sizes to answer this RQ (in
>>which case the original RQ must change too)?
>>
>>Thank you for helping me better conceptualize this basic question,
>>Luke