[R-meta] Question about my Effect Size Calculation

[Viechtbauer, Wolfgang (SP)]

Dear Sera,

Arguments sd1i and sd2i are really for the SDs of the raw data, so this isn't quite right.

If I understand you correctly, within each study, you have the means and SDs for the raw data corresponding to 4 conditions that subjects underwent. Let's call these means

m1, m2, m3, and m4

and the SDs

s1, s2, s3, and s4.

You eventually want to compute something like d = ((m1 - m2) - (m3 - m4)) / SD, where SD is some kind of standard deviation, the nature of which is to be determined.

The var-cov matrix of the raw measurements looks like this:

[s1^2       r12*s1*s2 r13*s1*s3 r14*s1*s4]
[r12*s1*s2 s2^2       r23*s2*s3 r24*s2*s4]
[                     s3^2      r34*s3*s4]
[                               s4^2     ]

The problem is that all those correlations are probably unknown, but you seem to assume r12=r34=0.7 and r13=r14=r23=r24=0.6.

Based on this, we know what the mean of the change scores was within the 1-2 and 3-4 conditions, namely:

mean(x1 - x2) = m1 - m2
mean(x3 - x4) = m3 - m4

and we know what the SD of these change scores was, namely:

SDx1x2 = SD(x1 - x2) = sqrt(s1^2 + s2^2 - 2*0.7*s1*s2)
SDx3x4 = SD(x3 - x4) = sqrt(s3^2 + s4^2 - 2*0.7*s3*s4)

So now you have everything to plug this into escalc():

escalc(measure = "SMCC", m1i = m1-m2, m2i = m3-m4, sd1i = SDx1x2, sd2i = SDx3x4, ni = n, ri = 0.6)

You can even do this within a single call to escalc(). Unless I got the variable names screwed up, this would be:

escalc(measure = "SMCC", 
       m1i  = NeutralCritical  - NeutralUnrelated, 
       m2i  = NegativeCritical - NegativeUnrelated,
       sd1i = sqrt(NeutralCriticalSD^2 + NeutralUnrelatedSD^2 - 
                   2*0.7*NeutralCriticalSD*NeutralUnrelatedSD),
       sd2i = sqrt(NegativeCritical^2 + NegativeUnrelated^2 - 
                   2*0.7*NegativeCritical*NegativeUnrelated),
       ni   = `Number PP`, 
       ri   = 0.6,
       data = Data)

(use the 'data' argument to make the code more legibile).

This will use the SD of the change scores in the denominator for computing d. Since sd1i and sd2i are SDs of change scores (for x1-x2 and x3-x4) already, this means that the denominator is the SD of (x1-x2)-(x3-x4).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Sera Wiechert
>Sent: Monday, 17 May, 2021 11:32
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Question about my Effect Size Calculation
>
>Dear R-sig-meta-analysis Community,
>
>Currently, I am in the process of conducting my own meta-analysis. For this, I
>need to use a slightly awkward effect size measure, namely a standardized
>difference of a standardized difference.
>
>To clarify a bit further - in the paradigm that I am investigating, they are
>comparing means between two groups of list conditions (critical vs unrelated). And
>I am additionally interested in the effect of negative valence (in comparison to
>neutral valence). So, I am looking at: neutral condition (critical mean -
>unrelated mean) - negative condition (critical mean- unrelated mean). For this, I
>tried to use the metafor package (for matched/paired designs).
>
>As I am using the effect size calculation twice to get to my "end" effect size, I
>am unsure on how to convert the variance output of the first effect size
>calculation into a SD to be able to use it again in the second calculation.
>
>I have now done it as follows:
>
>library(metafor)
>Data$Correlationwithin <- rep(0.7, nrow(Data))
>Data$Correlationbetweenemotion <- rep(0.6, nrow(Data))
>
>Neutral <- escalc(measure = "SMCC", m1i = Data$NeutralCritical, m2i =
>Data$NeutralUnrelated,
>       sd1i = Data$NeutralCriticalSD, sd2i = Data$NeutralUnrelatedSD, ni=
>Data$`Number PP`, ri = Data$Correlationwithin)
>
>
>##calculate effect sizes for negative: critical-unrelated
>Negative <- escalc(measure = "SMCC", m1i = Data$NegativeCritical, m2i =
>Data$NegativeUnrelated,
>                       sd1i = Data$NegativeCriticalSD, sd2i =
>Data$NegativeUnrelatedSD, ni= Data$`Number PP`, ri = Data$Correlationwithin)
>
>##transform variances to SEs
>for (i in 1:nrow(Neutral)){
>  Neutral$vi[i] <- sqrt(Neutral$vi[i])
>}
>
>for (i in 1:nrow(Negative)){
>  Negative$vi[i] <- sqrt(Negative$vi[i])
>}
>
>##transform SEs to SDs
>for (i in 1:nrow(Neutral)){
>  Neutral$vi[i] <- Neutral$vi[i] * sqrt(Data$`Number PP`[i])
>}
>
>for (i in 1:nrow(Negative)){
>  Negative$vi[i] <- Negative$vi[i] * sqrt(Data$`Number PP`[i])
>}
>
>##Calculate standardized emotion differences
>NeutNeg <- escalc(measure = "SMCC", m1i = Neutral$yi, m2i = Negative$yi,
>                       sd1i = Neutral$vi, sd2i = Negative$vi, ni= Data$`Number
>PP`, ri = Data$Correlationbetweenemotion)
>
>Would this work like this? Or is there an easier way to calculate my effect size
>measure of interest?
>
>Your help would be highly appreciated in this matter.
>Thanks in advance, and kind regards,
>Sera Wiechert