[R-meta] Percent Correct as outcome variable

[Viechtbauer Wolfgang (SP)]

Dear Markus,

Sure, you can compute D = M_X - M_Y. Since M_X and M_Y are means, this is a mean difference. This can be easily handled by metafor, meta, etc. In metafor, you can compute mean differences with escalc(measure="MD", ...) and then pass those to rma(). In meta, there is metacont(sm="MD", ...).

The only problem is that in order to compute the sampling variance of a mean difference, you need the within-group variances, since:

var(D) = Var(X) / n_X + Var(Y) / n_Y.

So, unless you impute the missing within-group variances, you cannot include such studies when using standard meta-analytic procedures.

If you do know the t-test statistic or p-value (from which one could back-calculate the t-statistic), then:

Var(pooled) = D^2 / (t^2 * (1/n_X + 1/n_Y))

so then you can back-calculate the pooled within-group variance (assuming the t-test computed was also computed based on the pooled variance) and then we could assume Var(pooled) = Var_X = Var_Y for computing var(D).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>project.org] On Behalf Of Markus Janczyk
>Sent: Monday, 29 January, 2018 11:05
>To: r-sig-meta-analysis at r-project.org
>Subject: [R-meta] Percent Correct as outcome variable
>
>Dear everybody,
>
>We are interested in a meta-analytic question related to memory research.
>
>Consider the case where two conditions X and Y are compared.
>Unfortunately, in many of the (old) papers only the means M_X and M_Y
>are reported for the percent correct remembered items (and often not
>even the interesting t-test). To calcutale a "real" effect size as the
>outcome measure to use in a meta-analysis, we need to some variability
>measure though (if I understood the metafor functions right). With the
>data we have, it feels like a raw (and non-standardized) effect we can
>caluculate as the outcome variable (i.e., D = M_X - M_Y).
>
>Is there any other possible solution or improvement somebody knows of or
>recommends or just somebody who can let me know a reference where I can
>look?
>
>Thanks, Markus
>
>--
>Jun.-Prof. Dr. phil. habil. Markus Janczyk, Dipl.-Psych.
>
>University of Tübingen
>Department of Psychology
>Cognition and Action
>Schleichstraße 4
>72076 Tübingen
>Germany
>
>http://www.pi.uni-tuebingen.de/arbeitsbereiche/kognition-und-
>handlung/research-group.html
>email: markus.janczyk at uni-tuebingen.de
>phone: +49 (0)7071 2976761