[R-meta] Question about Cohen's d and meta-regression

[Angeline Tsui]

Hello Wolfgang,

Thank you for your reply. Just a quick follow up question: I think that we
need to take into account of the correlation between pre-test and post-test
when we need to calculate the SD of difference score. I see this may be
important for my study because if there is a large treatment effect at
post-test, the correlation between pre-test and post-test would likely to
be small. In this case, it looks like that I should not use pre-test SD for
the SD of difference score. In this case, what is your recommendation?
Should I use post-test SD instead if SD of difference score is not provided?

Thanks,
Angeline

On Sun, Jan 7, 2018 at 5:24 PM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> A "within-subject Cohen's d" can be computed in two ways:
>
> 1) by dividing the mean difference by the SD of the change scores
> (standardized mean change using change score standardization) or
>
> 2) by dividing the mean difference by the SD of the raw scores
> (standardized mean change using raw score standardization). Here, the
> pre-test SD is often used (although one could also use the post-test SD or
> pool pre- and post-test SDs, although for the latter one needs to do some
> extra work to get a correct estimate of the sampling variance).
>
> The second approach yields a d value that is conceptually the same as a
> "between-subject Cohen's d" (if we 'equate' the pre condition with the
> control condition and the post condition with the experimental/treatment
> condition and assume that there are no time effects). So, I see no need for
> changing a within-subject Cohen's d to between-subject Cohen's d (leaving
> aside how that would actually be done).
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Angeline Tsui [mailto:angelinetsui at gmail.com]
> Sent: Thursday, 04 January, 2018 13:11
> To: Philipp Doebler; Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Question about Cohen's d and meta-regression
>
> Dear Wolfgang and Philipp,
>
> Thank you so much for the detailed response. This is exactly what I am
> looking for. I will look into the Knapp & Hartung correction and also do a
> power analysis of meta-regression. Thanks again for the recommended
> resources.
>
> Re Wolfgang as for the question about changing within-subject Cohen's d to
> between-subject Cohen's d: To be honest, I have struggled quite a lot
> before making this decision. The primary reason is due to recommendations
> from researchers in my field. They once advocated meta-analysis in my field
> to change within-subject Cohen's d to between-subject Cohen's d. I think it
> is simply related to converting meta-analysis mean effect size to a common
> metric, so that we can directly compare mean effect sizes across
> meta-analyses. Another reason (I think I may be wrong and please feel free
> to correct me) is that I was worried if I can still apply the Cohen's rule
> of thumb (0.2, 0.5, 0.8) to interpret within-subject Cohen's d. I know that
> this rule of thumb applies to between-subject Cohen's d, but I really am
> not sure if we can do this for within-subject Cohen's d. Specifically, the
> standard deviations are different between within-subject and
> between-subject designs. The standard deviations for within-subject design
> are theoretically smaller since we are controlling for individual
> differences. Thus, I changed my metric to within-subject Cohen's d just for
> the ease of interpretation as well.
>
> Best,
> Angeline
>



-- 
Best Regards,
Angeline

	[[alternative HTML version deleted]]