[R-meta] SMD Metric

[Viechtbauer, Wolfgang (NP)]

Measure "ROM" is not meant to be used for interval scale data. It is really meant to be used for data measured on a ratio scale, which has an absolute zero point and so the only permissible transformations are multiplicative ones (under which the values are again invariant):

x1 <- rlnorm(50, -1, 2)
x2 <- rlnorm(50, -2, 2)

escalc(measure="ROM", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
                      m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))

x1 <- x1 * 10
x2 <- x2 * 10

escalc(measure="ROM", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
                      m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))

So, ROM is appropriate for things like height, weight, distance, time, but not typically for the types of scales/questionnaires that we construct in the social sciences. For example, Likert-type items could be scored from 1-7 but we could just as well use 0-6 (and some scales do), which doesn't affect SMDs but would lead to different ROM values.

And yes, with correlations, strictly speaking, we also assume linear equatability.

In any case, since linear equatability (or with ROMs, multiplicative equatability) might not hold, it is useful (if sufficient data is available) to examine to what extent there are systematic differences between different ways of measuring the same underlying construct.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Yuhang Hu via R-sig-meta-analysis
>Sent: Monday, 03 April, 2023 3:16
>To: R meta
>Cc: Yuhang Hu
>Subject: Re: [R-meta] SMD Metric
>
>Oops, correcting a typo in the second instance of x2:
>
>x2 <- 40 + x2 * 3
>
>Thank you,
>Yuhang
>
>On Sun, Apr 2, 2023 at 6:02 PM Yuhang Hu <yh342 using nau.edu> wrote:
>
>> Thank you, James.
>>
>> I think we usually treat items on a test/instrument to be equivalent (all
>> other things equal), if they differ from one another by some additive and
>> multiplicative constants. We can probably extend this inter-item
>> equivalency concept to equate different tests (all other things equal).
>>
>> It seems some effect sizes (e.g., SMD and correlation coef.) take this
>> into account and 'work best' if such equivalence holds and work less well
>> as this equivalence weakens.
>>
>> But other effect sizes seem to be somewhat insensitive to such
>> equivalency. ROM seems to be one of them. But do you think that it is
>> desirable for ROM to not give the same estimates for when such an
>> equivalence, in fact, exists:
>>
>> x1 <- rnorm(50, 36, 6)
>> x2 <- rnorm(50, 33, 6)
>>
>> library(metafor)
>> escalc(measure="ROM", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
>>        m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))
>>
>> x1 <- 40 + x1 * 3
>> x2 <- 40 + x2 * 7
>>
>> escalc(measure="ROM", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
>>        m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))
>>
>> Many thanks,
>> Yuhang
>>
>> PS. With a lack of linear equatability, ROM may probably be preferred over
>> SMD. However, I wonder what to replace COR with, when there is a lack of
>> linear equatability? I would think in that case there is no alternative to
>> COR and we need to resort to adding methodological moderators.