[R] (Slight) calculation discrepancy in escalc (metafor package)

[Viechtbauer Wolfgang (STAT)]

The pooled standard deviation is calculated exactly how you describe it. There are, however, two differences between your approach and the way escalc() computes the SMD and corresponding sampling variance:

1) Instead of the equation for approximating the correction factor (J), escalc() uses the exact equation, namely:

gamma(mi/2)/(sqrt(mi/2)*gamma((mi-1)/2))

where mi = n1+n2-2. In your example, that is:

> mi <- 5+5-2
> gamma(mi/2)/(sqrt(mi/2)*gamma((mi-1)/2))
[1] 0.9027033

which is really only slightly different from the value you computed (0.903226) and hence, the corrected SMD value computed by escalc() and your computations essentially agree.

2) More importantly, the sampling variance (Vd) is computed in escalc() by plugging the *corrected* version of the SMD into the equation you gave and no further adjustment is made to Vd. This explains the more substantial difference you found between your computations of the sampling variance and the value returned by escalc(). This kind of difference will only be noticeable when the sample sizes are small (as in your example). 

One could debate which approach is "more" correct. Probably neither is appropriate here. The equation for the sampling variance given below is actually a large-sample approximation. For n1=n2=5, it's probably not accurate at all. An alternative would be to discard that equation altogether and instead use an unbiased estimate of the sampling variance, which can be obtained with:

escalc(measure="SMD", m1i=5,m2i=10,n1i=5,n2i=5,sd1i=1,sd2i=2,vtype="UB")

which gives a value of 1.0487, much bigger than either of the two values.

Best,

Wolfgang

--
Wolfgang Viechtbauer, Ph.D., Statistician
Department of Psychiatry and Psychology
School for Mental Health and Neuroscience
Faculty of Health, Medicine, and Life Sciences
Maastricht University, P.O. Box 616 (VIJV1)
6200 MD Maastricht, The Netherlands
+31 (43) 388-4170 | http://www.wvbauer.com
________________________________________
From: r-help-bounces at r-project.org [r-help-bounces at r-project.org] On Behalf Of Stephen Ban [stephen.ban at my.jcu.edu.au]
Sent: Wednesday, August 22, 2012 2:28 AM
To: r-help at r-project.org
Subject: [R] (Slight) calculation discrepancy in escalc (metafor package)

Hello,

I recently started using the metafor package (version 1.6-0) in R (2.15.1,
64-bit Windows 7) and noticed that I was getting slightly different values
when I manually calculated the standardized mean difference versus what
escalc was giving me. Here's a very simple example:

escalc(measure="SMD", m1i=5,m2i=10,n1i=5,n2i=5,sd1i=1,sd2i=2,vtype="LS")

The result is:

         yi        vi
1 -2.854599 0.8074367

However, if I calculate this manually using the pooled standard deviation
formula given in Cooper et al (2009)*, I get an SMD of -2.85625 and a
variance of 0.870205.

The formula given in Cooper et al for the pooled standard deviation is:

Swithin = sqrt(((n1i - 1) * Sd1i^2 + (n2i - 1) * Sd2i^2)/(n1i + n2i -2))
[1.581139 for my example]

And the formula for variance is:

Vd = (n1i + n2i)/(n1i*n2i) + d^2/(2 *(n1i+n2i)) [1.066667 for my example]

Where d is:

d = (m1i - m2i)/Swithin [-3.16228 for my example]

These are both corrected by the correction factor J:

J = 1 - 3/(4*(n1i+n2i-2)-1) [0.903226 for my example]

The resulting g is [0.903226 * -3.16228] = -2.85625 and the resulting
variance is [0.903226^2 * 1.06667] = 0.870205. Thus, my calculated SMD is
very close to the escalc value (only 0.05% different), but the variance is
nearly 8% different.

I assume the difference must lie in how the pooled standard deviation is
being calculated, but the help for escalc only says that the pooled standard
deviation is "calculated inside the function", and I can't find the exact
formula for how this is being done. I'm trying to track down a copy of
Hedges and Olkin (1985), but it would be great in the meantime somebody
could enlighten me.

Thanks,

Stephen Ban

*p. 226. Cooper, H., Hedges, L., and Valentine, J. The handbook of research
synthesis and metaanalysis. Russell Sage Foundation, 2009.

Stephen Ban, M.Sc.
Ph.D. Candidate
Australian Research Council Centre of Excellence for Coral Reef Studies
James Cook University
Townsville, Queensland 4811 Australia
Phone: +61 7 4781 6063
Fax: +61 7 4781 6722
 <mailto:stephen.ban at jcu.edu.au> stephen.ban at my.jcu.edu.au