[R-meta] SMD Metric

[Viechtbauer, Wolfgang (NP)]

Dear Yuhang,

Essentially, it means that the values on one instrument are assumed to be a linear transformation of the values on another instrument. For example, say we have measured two groups using scale/instrument/questionnaire A and we find:

x1 <- rnorm(50, 36, 6)
x2 <- rnorm(50, 33, 6)

library(metafor)
escalc(measure="SMD", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
                      m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))

Now imagine that instead of A, we had used another scale/instrument/questionnaire B and that the values on that instrument are simply a linear transformation of the scores that would have been obtained on A:

x1 <- 40 + x1 * 3
x2 <- 40 + x2 * 3

escalc(measure="SMD", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
                      m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))

As you can see, the SMD values are identical then.

So if values on different instruments are linearly equatable, then it doesn't matter if we use A or B, the 'effect size' would be identical.

But many scales/instruments/questionnaires do not exhibit such strict linear equatability. In that case, SMD values may be systematically higher/lower depending on the instrument used and we end up with a measurement artifact in our meta-analysis.

I hope that this clarifies things.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Yuhang Hu via R-sig-meta-analysis
>Sent: Sunday, 02 April, 2023 19:21
>To: R meta
>Cc: Yuhang Hu
>Subject: [R-meta] SMD Metric
>
>Hi Everyone,
>
>I had a question about the SMD effect size. I read on James' website that:
>
>"The ideal case for using the SMD metric is when the outcomes in different
>studies are linearly equatable. However, if outcomes exhibit mean-variance
>relationships, linearly equatability seems rather implausible."
>
>I was wondering what is meant by linear equatability in the outcomes in
>different studies and why is that needed for the use of SMD?  How could the
>outcomes in different studies be perhaps non-linearly equatable or not
>equatable at all (neither linearly nor non-linearly)?
>
>(I also appreciate reference(s) that discuss such a requirement for the use
>of the SMD metric)
>
>Thank you very much for your assistance,
>Yuhang