[R-meta] meta-analysis of response ratios with low sample sizes

[James Pustejovsky]

Ana,

The formulas for calculating response ratio effect size estimates and
sampling variances involve some asymptotic approximations---that is, they
will be about right if the sample size in each group is reasonably large.
With just two observations per group, I doubt that the approximations are
trustworthy, and the sampling distribution of the effect size estimate
could be pretty far from normal. There are small-sample bias corrections
available (Lajeunesse, 2015) but I would be concerned that even these would
break down with such small samples.

I could see two paths that might be appropriate in your situation:
1) Move towards a multi-variate, arm-based model, where the effect size is
simply the mean of the outcome within each group. Using a generalized
linear mixed model with a log link function would yield coefficients that
are interpretable as response ratios.
2) Stick with the model that you are currently using, but report
sensitivity analysis where studies with very small samples are excluded.

James

Lajeunesse, M. J. (2015). Bias and correction for the log response ratio in
ecological meta‐analysis. *Ecology*, *96*(8), 2056-2063.

On Fri, Jan 18, 2019 at 10:23 AM Michael Dewey <lists using dewey.myzen.co.uk>
wrote:

> Dear Ana
>
> Comments in-line
>
> On 18/01/2019 15:24, Ana Benitez wrote:
> > Dear Wolfgang (and users of the meta-analysis mailing list),
> >
> > I am currently conducting a meta-analisis where I want to assess body
> size
> > shifts in vertebrates living in islands compared to mainland populations
> > (a.k.a the island rule). I am using response ratios between the mean size
> > of the island population and the mean size of the mainland population. In
> > some cases I have measurements for only 2 specimens, and I calculate mean
> > and SD for those 2 specimens in order to calculate the sampling variance.
> > However, many people would argue that calculating the SD of 2 data points
> > is a bit meaningless in most contexts, but in a meta-analytical context I
> > would expect that response ratios based on N = 2 for either the treatment
> > or control, or both, would be downweighted in the metaanalysis and thus
> it
> > is both informative and interesting to include them in the analyses. I
> > would like to know if other people have encountered these situations and
> > how they dealt with it. Also, what’s your opinion, Wolfgang?
> >
>
> I think your feeling that (a) you can do it (b) they will be
> downweighted is correct.
>
> > I have a second query, in this same analysis I have cases where only one
> > specimen is measured, and thus the SD is zero. To be able to calculate
> the
> > sampling variance I add a small constant (0.5) to both the numerator and
> > denominator of the formula. Is this a sensible way to proceed or shall I
> > just discard cases where only 1 specimen is measured in either of the two
> > populations (or both of them)?
> >
>
> I do not like excluding anything but in this case I think it might be
> better than adding an arbitrary constant. If I was forced to add a
> constant by powerful figures then I would use a range of values to check
> whether the specific value I added was crucial. If it is then I would be
> even more doubtful about the wisdom of adding it.
>
> Michael
>
> > Thanks a lot for your time, I am looking forward to your thoughts on
> these
> > two queries.
> >
> > Best,
> >
> > Ana
> >
>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]