[R-meta] mean-variance relationships introduces additional heterogeneity, how?

[Luke Martinez]

Sure. All of this seems to imply that the use of AN(C)OVAs and t-tests
for proportions and counts is similarly (and perhaps even more
severely) affected by the mean-sd relationship that exists in most
count distributions. So, researchers, one would say, should use glm()s
or glmms instead. But again in that context still means and sds of
those counts and proportions is useful to perhaps compute the LRRs.

On the list, I found this study
(https://onlinelibrary.wiley.com/doi/epdf/10.1002/sim.9226) but I
wonder which equation in it gives the sampling variance of LRR in
2-level designs, possibly Equation (3)?

If yes, how to compute Var(C_bar)?

Luke


On Mon, Oct 25, 2021 at 9:07 PM James Pustejovsky <jepusto using gmail.com> wrote:
>
> All I mean is that a skewed distribution or one with large outliers
> does not necessarily *imply* that a mean-sd relationship exists. It
> could be the result of one, but skewness might be due to something
> else (such as selective reporting) instead.
>
> I would suggest that a well-behaved effect distribution is desirable
> and appropriate to the extent that it indicates empirical regularity
> of the phenomenon you're interested in. A less heterogeneous
> distribution means that effects are more predictable (at least in the
> corpus of studies that you're examining).
>
> On Mon, Oct 25, 2021 at 8:58 PM Luke Martinez <martinezlukerm using gmail.com> wrote:
> >
> > I thought the existence of outlying effect estimates under SMD and
> > lack of it under LRR could attest to the existence of
> > heterogeneity-generating artefacts like mean-sd relationships (and/or
> > variation in measurement error) across the studies.
> >
> > If not, then, would you mind commenting on why a more symmetric and
> > well-behaved effect distribution is equated with its appropriateness
> > for a set of summaries (e.g., means & sds) from studies?
> >
> > Luke
> >
> > On Mon, Oct 25, 2021 at 8:47 PM James Pustejovsky <jepusto using gmail.com> wrote:
> > >
> > > Responses below.
> > >
> > > On Mon, Oct 25, 2021 at 4:21 PM Luke Martinez <martinezlukerm using gmail.com> wrote:
> > > >
> > > > Sure, thanks. Along the same lines, if I see that the unconditional
> > > > distribution of the SMD estimates is multi-modal or right or left
> > > > skewed (perhaps due to extreme outliers), but the unconditional
> > > > distribution of the corresponding LRR estimates looks more symmetric
> > > > and well-behaved, does that also empirically suggest a mean-sd
> > > > relationship in one or more groups?
> > >
> > > I'm not sure that it implies a mean-sd relationship. But I think it
> > > does suggest that LRR might be a more appropriate metric.
> > >
> > > > PS. Is there a reason for exploring the mean-sd relationship
> > > > specifically in the control group?
> > >
> > > No, you could certainly examine the relationships in the treatment
> > > group(s) as well.