[R-meta] Question about Cohen's d and meta-regression

[Angeline Tsui]

Dear Wolfgang,

Thank you very much for your reply. I apologize. I had a wrong
interpretation of correlation between mean change. I checked the equation
again, yes, you are absolutely correct that the small correction simply
mean that there is a lot of variability of mean change. So I think I have
solved my question because there is no differences of using either approach
to computer a standardized mean change.

Thank you again for your kind explanation and time on my question. I
greatly appreciate it.

Best,
Angeline

On Wed, Jan 17, 2018 at 3:47 AM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> Hi Angeline,
>
> I don't quite understand what your concern is. Neither approach (of
> computing a standardized mean change) is right or wrong -- they are just
> different ways of standardizing the mean change. Also, a large treatment
> effect does not automatically translate into a small correlation. If the
> effect is such that the mean goes up (or down) by a large constant for
> everybody, then the correlation will be 1. It is only when there is large
> variability in the size of the treatment effect that the correlation will
> be small. But again, I am not able to follow what you are concerned about,
> so I cannot really answer your question.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Angeline Tsui [mailto:angelinetsui at gmail.com]
> Sent: Monday, 08 January, 2018 0:15
> To: Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Question about Cohen's d and meta-regression
>
> Hello Wolfgang,
>
> Thank you for your reply. Just a quick follow up question: I think that we
> need to take into account of the correlation between pre-test and post-test
> when we need to calculate the SD of difference score. I see this may be
> important for my study because if there is a large treatment effect at
> post-test, the correlation between pre-test and post-test would likely to
> be small. In this case, it looks like that I should not use pre-test SD for
> the SD of difference score. In this case, what is your recommendation?
> Should I use post-test SD instead if SD of difference score is not provided?
>
> Thanks,
> Angeline
>
> On Sun, Jan 7, 2018 at 5:24 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> A "within-subject Cohen's d" can be computed in two ways:
>
> 1) by dividing the mean difference by the SD of the change scores
> (standardized mean change using change score standardization) or
>
> 2) by dividing the mean difference by the SD of the raw scores
> (standardized mean change using raw score standardization). Here, the
> pre-test SD is often used (although one could also use the post-test SD or
> pool pre- and post-test SDs, although for the latter one needs to do some
> extra work to get a correct estimate of the sampling variance).
>
> The second approach yields a d value that is conceptually the same as a
> "between-subject Cohen's d" (if we 'equate' the pre condition with the
> control condition and the post condition with the experimental/treatment
> condition and assume that there are no time effects). So, I see no need for
> changing a within-subject Cohen's d to between-subject Cohen's d (leaving
> aside how that would actually be done).
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Angeline Tsui [mailto:angelinetsui at gmail.com]
> Sent: Thursday, 04 January, 2018 13:11
> To: Philipp Doebler; Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Question about Cohen's d and meta-regression
>
> Dear Wolfgang and Philipp,
>
> Thank you so much for the detailed response. This is exactly what I am
> looking for. I will look into the Knapp & Hartung correction and also do a
> power analysis of meta-regression. Thanks again for the recommended
> resources.
>
> Re Wolfgang as for the question about changing within-subject Cohen's d to
> between-subject Cohen's d: To be honest, I have struggled quite a lot
> before making this decision. The primary reason is due to recommendations
> from researchers in my field. They once advocated meta-analysis in my field
> to change within-subject Cohen's d to between-subject Cohen's d. I think it
> is simply related to converting meta-analysis mean effect size to a common
> metric, so that we can directly compare mean effect sizes across
> meta-analyses. Another reason (I think I may be wrong and please feel free
> to correct me) is that I was worried if I can still apply the Cohen's rule
> of thumb (0.2, 0.5, 0.8) to interpret within-subject Cohen's d. I know that
> this rule of thumb applies to between-subject Cohen's d, but I really am
> not sure if we can do this for within-subject Cohen's d. Specifically, the
> standard deviations are different between within-subject and
> between-subject designs. The standard deviations for within-subject design
> are theoretically smaller since we are controlling for individual
> differences. Thus, I changed my metric to within-subject Cohen's d just for
> the ease of interpretation as well.
>
> Best,
> Angeline
>



-- 
Best Regards,
Angeline

	[[alternative HTML version deleted]]