[R-meta] Question about Cohen's d and meta-regression

[Viechtbauer Wolfgang (SP)]

A "within-subject Cohen's d" can be computed in two ways:

1) by dividing the mean difference by the SD of the change scores (standardized mean change using change score standardization) or

2) by dividing the mean difference by the SD of the raw scores (standardized mean change using raw score standardization). Here, the pre-test SD is often used (although one could also use the post-test SD or pool pre- and post-test SDs, although for the latter one needs to do some extra work to get a correct estimate of the sampling variance).

The second approach yields a d value that is conceptually the same as a "between-subject Cohen's d" (if we 'equate' the pre condition with the control condition and the post condition with the experimental/treatment condition and assume that there are no time effects). So, I see no need for changing a within-subject Cohen's d to between-subject Cohen's d (leaving aside how that would actually be done).

Best,
Wolfgang

-----Original Message-----
From: Angeline Tsui [mailto:angelinetsui at gmail.com] 
Sent: Thursday, 04 January, 2018 13:11
To: Philipp Doebler; Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Question about Cohen's d and meta-regression

Dear Wolfgang and Philipp,

Thank you so much for the detailed response. This is exactly what I am looking for. I will look into the Knapp & Hartung correction and also do a power analysis of meta-regression. Thanks again for the recommended resources.

Re Wolfgang as for the question about changing within-subject Cohen's d to between-subject Cohen's d: To be honest, I have struggled quite a lot before making this decision. The primary reason is due to recommendations from researchers in my field. They once advocated meta-analysis in my field to change within-subject Cohen's d to between-subject Cohen's d. I think it is simply related to converting meta-analysis mean effect size to a common metric, so that we can directly compare mean effect sizes across meta-analyses. Another reason (I think I may be wrong and please feel free to correct me) is that I was worried if I can still apply the Cohen's rule of thumb (0.2, 0.5, 0.8) to interpret within-subject Cohen's d. I know that this rule of thumb applies to between-subject Cohen's d, but I really am not sure if we can do this for within-subject Cohen's d. Specifically, the standard deviations are different between within-subject and between-subject designs. The standard deviations for within-subject design are theoretically smaller since we are controlling for individual differences. Thus, I changed my metric to within-subject Cohen's d just for the ease of interpretation as well.

Best,
Angeline