[R-meta] Observed effect size using escalc

[Viechtbauer Wolfgang (SP)]

Forgot the closing parentheses on the equation for the bias correction factor. It should be:

exp(lgamma(mi/2) - log(sqrt(mi/2)) - lgamma((mi-1)/2))

Also, usually the equation is written as:

gamma(mi/2) / (sqrt(mi/2) * gamma((mi-1)/2))

but this can lead to numerical overflow when mi is large. An example:

mi <- 500
gamma(mi/2) / (sqrt(mi/2) * gamma((mi-1)/2))
exp(lgamma(mi/2) - log(sqrt(mi/2)) - lgamma((mi-1)/2))

Often, the following approximation is used:

1 - 3 / (4*mi - 1)

which works very well, even when mi is small.

Best,
Wolfgang

-----Original Message-----
From: Myers, Brett [mailto:brett.myers at vanderbilt.edu] 
Sent: Friday, August 11, 2017 23:52
To: Viechtbauer Wolfgang (SP); r-sig-meta-analysis at r-project.org
Subject: RE: [R-meta] Observed effect size using escalc

Thank you so much for that clarification! I really appreciate it.

Brett

________________________________________
From: Viechtbauer Wolfgang (SP) [wolfgang.viechtbauer at maastrichtuniversity.nl]
Sent: Friday, August 11, 2017 12:35 PM
To: r-sig-meta-analysis at r-project.org
Cc: Myers, Brett
Subject: RE: [R-meta] Observed effect size using escalc

For measure="SMD", escalc() computes:

ni <- n1i + n2i
mi <- ni - 2
sdpi <- sqrt(((n1i-1)*sd1i^2 + (n2i-1)*sd2i^2)/mi)
di <- (m1i - m2i) / sdpi
cmi <- .cmicalc(mi)
yi <- cmi * di

where .cmicalc() is a function that computes the bias correction factor, which is given by

exp(lgamma(mi/2) - log(sqrt(mi/2)) - lgamma((mi-1)/2)

The reference for this is Hedges (1981).

Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.

But, based on the naming of the variables, it is pretty clear that the escalc() function is being misused here. For m1i and m2i, mean changes are provided but pre (treatment?) SDs for sd1i and sd2i. This invalidates the assumption underlying the computations.

If you are dealing with two-group pre-post designs, then you will want to use measures like "SMCC" or "SMCR" (computed for each group separately) and then taking the difference as the effect size. See also:

http://www.metafor-project.org/doku.php/analyses:morris2008

Best,
Wolfgang

--
Wolfgang Viechtbauer, Ph.D., Statistician | Department of Psychiatry and
Neuropsychology | Maastricht University | P.O. Box 616 (VIJV1) | 6200 MD
Maastricht, The Netherlands | +31 (43) 388-4170 | http://www.wvbauer.com

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Myers, Brett
Sent: Friday, August 11, 2017 18:58
To: Michael Dewey; r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Observed effect size using escalc

Michael,

Thank you for your reply. Here is the pertinent line of code from which yi is derived:

meta_escalc <- escalc(measure="SMD", n1i=n, n2i=n.1, m1i=(post_mean-pre_mean), m2i=(post_mean.1-pre_mean.1), sd1i=pre_sd, sd2i=pre_sd.1, data=newdata)

In the summary data output, yi is introduced, so I assume escalc is calculating the effect size based on our input variables. I was under the impression effect size would be calculated as:
ES = [(post_mean-pre_mean) - (post_mean.1-pre_mean.1)] / (pooled pretest sd)

However, the solution of ES by hand does not match the yi output from escalc, which leads me to wonder what the calculation actually is. Any thoughts on this?

Brett