[R-meta] Computing Effect Size for Difference in Differences with Different Populations

[Mika Manninen]

Dear community,

I am currently working on a meta-analysis that aims to examine the
difference in training effects between two populations. Both
populations underwent the same training, but at pre-test, the groups
have significantly different means and standard deviations (about
1-2sd difference in means).

I am interested in computing the effect size for the difference in
differences between the two groups. Specifically, I would like to know
what is the best way to calculate the effect size given the
significant difference in means and standard deviations at pre-test.

Would the below be roughly accurate (Option 1):

Option 1.

G1 <- escalc(measure="SMCRH", m1i=postm_G1, m2i=prem_G1,
sd1i=postsd_G1,ni=n_G1, sd2i = presd_G1, ri=c(rep(0.7,10)), data=G)
G2 <- escalc(measure="SMCRH", m1i=postm_G2, m2i=prem_G2,
sd1i=postsd_G2, ni=n_G2, sd2i = presd_G2, ri=c(rep(0.7,10)), data=G)
dat <- data.frame(yi = G1$yi - G2$yi, vi = G1$vi + G2$vi)

Option 2.

ES = (G1 post_mean - G2 pre_mean) - (G2 post_mean - G2 pre_mean) / pldpre_sd
pldpre_sd = sqrt((presdG1^2 + presdG2^2) / 2)

### dataset

set.seed(123)

postm_G1 <- rnorm(100, mean = 14, sd = 2.5)
prem_G1 <- rnorm(100, mean = 10, sd = 2)
postsd_G1 <- rnorm(100, mean = 1.4, sd = 0.2)
presd_G1 <- rnorm(100, mean = 1, sd = 0.2)
n_G1 <- rpois(100, lambda = 50)

postm_G2 <- rnorm(100, mean = 7.5, sd = 1.8)
prem_G2 <- rnorm(100, mean = 5, sd = 1.2)
postsd_G2 <- rnorm(100, mean = 0.9, sd = 0.2)
presd_G2 <- rnorm(100, mean = 0.6, sd = 0.2)
n_G2 <- rpois(100, lambda = 50)

G <- data.frame(postm_G1, prem_G1, postsd_G1, n_G1, presd_G1,
postm_G2, prem_G2, postsd_G2, n_G2, presd_G2)

###

Thank you in advance for your time and help.

Best wishes,
Mika