[R-meta] A rather general question study/effect size ratio

[Farzad Keyhan]

Good point, Michael.

However, I was thinking of this more from the perspective of sampling.
The existence of multiple effects per study (scenarios A & B) may be
taken to indicate that the estimates have been sampled under a
multistage sampling plan from a larger population of studies (i.e.,
first, studies were randomly sampled, then, from within them, some
effects were randomly sampled, under a 3-level MLMA model).

The existence of a single effect per study (scenario C) may be taken
to indicate that the estimates have been sampled under a simple random
sampling plan from a larger population of studies (i.e., effects were
equally likely to be randomly sampled from any study in the population
of studies).

It would seem to me that generally the more the studies the better our
overall sense of the overall effect in the full population of studies.

Of the three scenarios (C) is the trickiest to me. It's a tough call
to say because each study has provided a single effect, then effects
were equally likely to be randomly sampled from any study in the
population of studies. Maybe if that had been the case, we would have
seen some studies with more than a single effect.

Perhaps all of this takes us back to using some form of
robust/sandwich estimation of effects, even if we have a single effect
per study.

Fred

On Fri, Feb 4, 2022 at 11:19 AM Michael Dewey <lists using dewey.myzen.co.uk> wrote:
>
> Dear Farzad
>
> If having more studies means that the treatment has been tried in a
> wider range of settings then my vote is for more studies rather than
> more effects per study. If the settings are homogeneous but the outcomes
> are varied then more outcomes might improve generalisability.
>
> Michael
>
> On 04/02/2022 16:50, Farzad Keyhan wrote:
> > Hello All,
> >
> > All else equal, would it be more desirable to have (A) 40 studies each
> > with two estimates of effect size or (B) 20 studies each with 4
> > estimates of effect size?
> >
> > My intuition is that (A) is more desirable than (B).
> >
> > Would it also be more desirable to have (C) 80 studies each with 1
> > estimate of effect size over (A) and (B)?
> >
> > ps. By "desirable" I mean higher generalizability to the target population.
> >
> > Thank you,
> > Fred
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list
> > R-sig-meta-analysis using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >
>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html