[R-meta] SMD Metric

[Yuhang Hu]

Dear Wolfgang,

Thank you so much for your response. I would imagine that linear
equatability is likely required for the use of many other effect sizes
(e.g., correlation coefficients), right?

But is/are there possibly some reference(s) discussing the 'linear
equatability requirement' for the use of SMD or any other effect sizes (or
perhaps any additional considerations like the ones I mention below)?

You noted that "But many scales/instruments/questionnaires do not exhibit
such strict linear equatability".

I wonder what are the underlying reasons for that? For instance, the lack
of linear equatability is because the instrument across studies (A) could
target slightly different constructs (so their latent constructs differs in
location and scale by a bit), or (B) they differ in length or time allowed
to respond to the items (and thus in reliability), or (C) the items across
the instruments differ in degrees of item difficulty and discrimination, or
perhaps (D) the items across the instruments differ in their scale of
measurement (one binary, another Likert scale etc.) and thus respondents'
responses to the items across the instruments are distributed differently
(one binomially distributed, another ordered-categorically distributed etc.)

Thank you again, for your help,
Yuhang

On Sun, Apr 2, 2023 at 10:56 AM Viechtbauer, Wolfgang (NP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Yuhang,
>
> Essentially, it means that the values on one instrument are assumed to be
> a linear transformation of the values on another instrument. For example,
> say we have measured two groups using scale/instrument/questionnaire A and
> we find:
>
> x1 <- rnorm(50, 36, 6)
> x2 <- rnorm(50, 33, 6)
>
> library(metafor)
> escalc(measure="SMD", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
>                       m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))
>
> Now imagine that instead of A, we had used another
> scale/instrument/questionnaire B and that the values on that instrument are
> simply a linear transformation of the scores that would have been obtained
> on A:
>
> x1 <- 40 + x1 * 3
> x2 <- 40 + x2 * 3
>
> escalc(measure="SMD", m1i=mean(x1), sd1i=sd(x1), n1i=length(x1),
>                       m2i=mean(x2), sd2i=sd(x2), n2i=length(x2))
>
> As you can see, the SMD values are identical then.
>
> So if values on different instruments are linearly equatable, then it
> doesn't matter if we use A or B, the 'effect size' would be identical.
>
> But many scales/instruments/questionnaires do not exhibit such strict
> linear equatability. In that case, SMD values may be systematically
> higher/lower depending on the instrument used and we end up with a
> measurement artifact in our meta-analysis.
>
> I hope that this clarifies things.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Yuhang Hu via R-sig-meta-analysis
> >Sent: Sunday, 02 April, 2023 19:21
> >To: R meta
> >Cc: Yuhang Hu
> >Subject: [R-meta] SMD Metric
> >
> >Hi Everyone,
> >
> >I had a question about the SMD effect size. I read on James' website that:
> >
> >"The ideal case for using the SMD metric is when the outcomes in different
> >studies are linearly equatable. However, if outcomes exhibit mean-variance
> >relationships, linearly equatability seems rather implausible."
> >
> >I was wondering what is meant by linear equatability in the outcomes in
> >different studies and why is that needed for the use of SMD?  How could
> the
> >outcomes in different studies be perhaps non-linearly equatable or not
> >equatable at all (neither linearly nor non-linearly)?
> >
> >(I also appreciate reference(s) that discuss such a requirement for the
> use
> >of the SMD metric)
> >
> >Thank you very much for your assistance,
> >Yuhang
>

	[[alternative HTML version deleted]]