[R-meta] mean-variance relationships introduces additional heterogeneity, how?

[Luke Martinez]

Hi James,

Thanks a lot for investing so much effort into my question! Let me ask
a quick question regarding the second diagnostic in your post.

In your post, you note that *"[Since] there is a strong linear
relationship between the two [groups'] means, with a best-fit line
that might go through the origin. . . the response ratio might be an
appropriate metric."*

Could you please elaborate on how this speaks to the appropriateness
of LRR over SMD?

Luke

On Tue, Nov 2, 2021 at 8:12 AM James Pustejovsky <jepusto using gmail.com> wrote:
>
> HI Luke and listserv,
>
> I wrote up some thoughts on the question of using standardized mean
> differences to analyze outcomes measured as proportions:
> https://www.jepusto.com/mean-variance-relationships-and-smds/
> Thoughts, comments, questions, and critiques welcome.
>
> James
>
> On Mon, Oct 25, 2021 at 9:07 PM James Pustejovsky <jepusto using gmail.com> wrote:
> >
> > All I mean is that a skewed distribution or one with large outliers
> > does not necessarily *imply* that a mean-sd relationship exists. It
> > could be the result of one, but skewness might be due to something
> > else (such as selective reporting) instead.
> >
> > I would suggest that a well-behaved effect distribution is desirable
> > and appropriate to the extent that it indicates empirical regularity
> > of the phenomenon you're interested in. A less heterogeneous
> > distribution means that effects are more predictable (at least in the
> > corpus of studies that you're examining).
> >
> > On Mon, Oct 25, 2021 at 8:58 PM Luke Martinez <martinezlukerm using gmail.com> wrote:
> > >
> > > I thought the existence of outlying effect estimates under SMD and
> > > lack of it under LRR could attest to the existence of
> > > heterogeneity-generating artefacts like mean-sd relationships (and/or
> > > variation in measurement error) across the studies.
> > >
> > > If not, then, would you mind commenting on why a more symmetric and
> > > well-behaved effect distribution is equated with its appropriateness
> > > for a set of summaries (e.g., means & sds) from studies?
> > >
> > > Luke
> > >
> > > On Mon, Oct 25, 2021 at 8:47 PM James Pustejovsky <jepusto using gmail.com> wrote:
> > > >
> > > > Responses below.
> > > >
> > > > On Mon, Oct 25, 2021 at 4:21 PM Luke Martinez <martinezlukerm using gmail.com> wrote:
> > > > >
> > > > > Sure, thanks. Along the same lines, if I see that the unconditional
> > > > > distribution of the SMD estimates is multi-modal or right or left
> > > > > skewed (perhaps due to extreme outliers), but the unconditional
> > > > > distribution of the corresponding LRR estimates looks more symmetric
> > > > > and well-behaved, does that also empirically suggest a mean-sd
> > > > > relationship in one or more groups?
> > > >
> > > > I'm not sure that it implies a mean-sd relationship. But I think it
> > > > does suggest that LRR might be a more appropriate metric.
> > > >
> > > > > PS. Is there a reason for exploring the mean-sd relationship
> > > > > specifically in the control group?
> > > >
> > > > No, you could certainly examine the relationships in the treatment
> > > > group(s) as well.