[R-meta] Percent Correct as outcome variable

[Markus Janczyk]

Thanks Wolfgang, it is exactly the problem that moste often no 
within-group variances were reported, nor were the relevant t-tests to 
back-calculate the variance... Is there still a method for a reasonable 
imputation of the variance? Something like a best guess? We know the 
sample size of course, sometimes the number of items per participants 
that was administered and could be right or wrong rememberd.

Best, Markus


Am 04.02.2018 um 13:33 schrieb Viechtbauer Wolfgang (SP):
> Dear Markus,
>
> Sure, you can compute D = M_X - M_Y. Since M_X and M_Y are means, this is a mean difference. This can be easily handled by metafor, meta, etc. In metafor, you can compute mean differences with escalc(measure="MD", ...) and then pass those to rma(). In meta, there is metacont(sm="MD", ...).
>
> The only problem is that in order to compute the sampling variance of a mean difference, you need the within-group variances, since:
>
> var(D) = Var(X) / n_X + Var(Y) / n_Y.
>
> So, unless you impute the missing within-group variances, you cannot include such studies when using standard meta-analytic procedures.
>
> If you do know the t-test statistic or p-value (from which one could back-calculate the t-statistic), then:
>
> Var(pooled) = D^2 / (t^2 * (1/n_X + 1/n_Y))
>
> so then you can back-calculate the pooled within-group variance (assuming the t-test computed was also computed based on the pooled variance) and then we could assume Var(pooled) = Var_X = Var_Y for computing var(D).
>
> Best,
> Wolfgang
>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>> project.org] On Behalf Of Markus Janczyk
>> Sent: Monday, 29 January, 2018 11:05
>> To: r-sig-meta-analysis at r-project.org
>> Subject: [R-meta] Percent Correct as outcome variable
>>
>> Dear everybody,
>>
>> We are interested in a meta-analytic question related to memory research.
>>
>> Consider the case where two conditions X and Y are compared.
>> Unfortunately, in many of the (old) papers only the means M_X and M_Y
>> are reported for the percent correct remembered items (and often not
>> even the interesting t-test). To calcutale a "real" effect size as the
>> outcome measure to use in a meta-analysis, we need to some variability
>> measure though (if I understood the metafor functions right). With the
>> data we have, it feels like a raw (and non-standardized) effect we can
>> caluculate as the outcome variable (i.e., D = M_X - M_Y).
>>
>> Is there any other possible solution or improvement somebody knows of or
>> recommends or just somebody who can let me know a reference where I can
>> look?
>>
>> Thanks, Markus
>>
>> --
>> Jun.-Prof. Dr. phil. habil. Markus Janczyk, Dipl.-Psych.
>>
>> University of Tübingen
>> Department of Psychology
>> Cognition and Action
>> Schleichstraße 4
>> 72076 Tübingen
>> Germany
>>
>> http://www.pi.uni-tuebingen.de/arbeitsbereiche/kognition-und-
>> handlung/research-group.html
>> email: markus.janczyk at uni-tuebingen.de
>> phone: +49 (0)7071 2976761

-- 
Jun.-Prof. Dr. phil. habil. Markus Janczyk, Dipl.-Psych.

University of Tübingen
Department of Psychology
Cognition and Action
Schleichstraße 4
72076 Tübingen
Germany

http://www.pi.uni-tuebingen.de/arbeitsbereiche/kognition-und-handlung/research-group.html
email: markus.janczyk at uni-tuebingen.de
phone: +49 (0)7071 2976761