[R-meta] effect size estimates distribution and field-specific benchmarks

[Yefeng Yang]

Dear community,

I have a question about the effect size distribution. It would be great if you would like to share your wisdom or just comment on it.

I briefly describe my question as follows:

I have a collection of effect size estimates of a specific field, say using SMD. Somehow, the dataset is free of publication bias. Now I want to derive the empirical benchmarks to inform the magnitude of the effect size estimates.
I am clear about the pitfalls of using empirical benchmarks because the interpretation of effect size should be specific to the context of the question/field. But for now, let's discuss the technical approaches to get reliable benchmarks.

At the moment, I am using four approaches:

  1.  using the empirical distribution to get the relevant percentiles, say 25, 50, 75th
  2.  using the mixture model to approximate the distribution and get relevant percentiles
  3.  fitting a meta-analysis model to get mean and variance, and then recover the normal distribution to get the relevant percentiles
  4.  fitting a Bayesian MA and get the posterior distribution

My purpose is to see whether different approaches converge in terms of the benchmarks (or more precisely, percentiles). Do you have any other approaches? General comments or suggestions are also welcome.

Regards,
Yefeng

	[[alternative HTML version deleted]]