[R-meta] Estimating "overall effect" in meta-regression

[Viechtbauer, Wolfgang (SP)]

Yes, that would make sense.

Best,
Wolfgang

>-----Original Message-----
>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>Sent: Monday, 10 January, 2022 17:47
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: Estimating "overall effect" in meta-regression
>
>Thank you so much, Wolfgang. I gather from your response that any
>additional moderators in such a meta-analysis must likewise interact
>with time so we can separate the effect of such a moderator at each
>different time point.
>
>Thank you very much,
>Luke
>
>On Mon, Jan 10, 2022 at 9:15 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>> For such a pre-post-control-group design design, one could compute two
>standardized mean differences (one for time 1 and one for time 2), or two
>standardized mean changes (one for the treatment and one for the control group),
>or one could compute the difference between two such estimates.
>>
>> Based on the schema below, it looks like you are computing two standardized
>mean differences (one for time 1 and one for time 2). Note that the two
>standardized mean differences are not independent, since they are computed based
>on the same subjects. This aside, then one definitely should include time as a
>moderator, as it would make little sense to synthesize estimates from before and
>after the treatment into a single pooled effect.
>>
>> Best,
>> Wolfgang
>>
>> >-----Original Message-----
>> >From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>> >Sent: Monday, 10 January, 2022 13:38
>> >To: Viechtbauer, Wolfgang (SP)
>> >Cc: R meta
>> >Subject: Re: Estimating "overall effect" in meta-regression
>> >
>> >Dear Wolfgang,
>> >
>> >The studies follow a pre-post-control design. The effect size measure used is
>> >standardized mean difference (SMD).
>> >
>> >I hope this clarifies my question.
>> >Luke
>> >
>> >sudy time  yi
>> >1         0
>> >1         1
>> >2         0
>> >2         1
>> >
>> >On Mon, Jan 10, 2022, 5:43 AM Viechtbauer, Wolfgang (SP)
>> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> >Dear Luke,
>> >
>> >I don't understand the question, in part because it is not clear to me what
>kind
>> >of design the studies have and what kind of effect size measure you are using.
>> >Could you clarify this?
>> >
>> >Best,
>> >Wolfgang
>> >
>> >>-----Original Message-----
>> >>From: Luke Martinez [mailto:martinezlukerm using gmail.com]
>> >>Sent: Sunday, 09 January, 2022 7:53
>> >>To: R meta
>> >>Cc: Viechtbauer, Wolfgang (SP)
>> >>Subject: Estimating "overall effect" in meta-regression
>> >>
>> >>Dear R-meta Community,
>> >>
>> >>I'm meta-analyzing a group of pre-post studies. My first RQ is: what
>> >>is the "overall effect" of policy X on a dependent variable.
>> >>
>> >>I know that I can fit an intercept-only model to answer this RQ.
>> >>
>> >>But an intercept-only model estimates the average effect size across
>> >>BOTH pre-tests (before policy X implementation) and post-tests (after
>> >>policy X implementation), while pre-test effect sizes don't contain
>> >>any policy X effect.
>> >>
>> >>Given that, is it appropriate to use an intercept-only model to answer
>> >>this RQ, or I actually need to use a time indicator as a moderator to
>> >>separate the pre- from post-test effect sizes to answer this RQ (in
>> >>which case the original RQ must change too)?
>> >>
>> >>Thank you for helping me better conceptualize this basic question,
>> >>Luke