[R-meta] Secon order meta-analysis

[Giovanni Tamburini]

Dear All,

I am working on a comprehensive review regarding the effect of sustainable
agriculture on different ecological processes. The study is solely based on
published meta-analyses (84 in total). Since a subset of these
meta-analyses present comparable effect sizes (54 papers, RR or LnRR, 255
individual analyses) I would like to perform a second order meta-analysis.
However, I found very few examples of this kind of approach in the
literature, especially for non-medical studies.

From each selected meta-analysis I have extracted mean effect size,
confidential intervals and sample size (e.g. number of original
comparisons). I transformed the effect size to log response ratio when
necessary and then I calculated the sampling error variance as the square
of the SE (SE= half the width of the 95% CI divided by 1.96). I excluded
those meta analyses with high proportion of shared original publications
(number of original studies varies from 14 to 602).

My current approach is to use classic statistics following examples from
Koricheva et al. (2013) for first order meta-analyses and Castellanos &
Verdú (2012),  the only ecological study I found applying this kind of
approach. I utilized mixed effect models (lme function), with the study as
random factor and setting both the sample size and the inverse of the
sampling error variance as weights (in two different models to be compared;
“weights” argument).

*My question is:* *Would it be possible to run a proper second order
meta-analysis (rma.mv <http://rma.mv>) with the (scarce) information
available?*

I have three main doubts regarding these potential analyses:

1. Should I put the sample size (e.g. number of original comparisons) as an
additional weight in the model (W)? In a normal meta-analysis this is
considered when calculating the sampling error variance, but in this case
it is only “hidden” in the mean effect size and correlated CI. The sample
size in my dataset varies greatly, from 3 to 5463. Trying to explain this
better: in my dataset some records present very similar mean effect sizes
with very similar CI but with very different sample size. Shouldn’t the
ones with higher sample size weight more than those with lower sample size?
2. Is it conceptually correct to analyse together effect sizes that
represent different ecological processes? All the effect sizes represent
changes in processes in response to management, but they greatly vary, from
changes in pollinator abundance to changes in soil organic carbon content
or CO2 emissions. It would be interesting and informative for me to
understand whether there is a communal response and hence analyzing them
together. I found a recent meta-analysis that did exactly the same (Winter
et al. 2017 Journal of Applied Ecology).
3. (Very) preliminary analyses show some publication bias (significant
Egger test or low fail safe-number). Is there any differences in
considering publication bias between meta-analysis and second order
meta-analysis?


I would be extremely grateful if you could help me with these issues! Thank
you very much in advance for your time and attention!

Best,

Giovanni Tamburini


References
Castellanos, M. C., & Verdú, M. (2012). Meta-analysis of meta-analyses in
plant evolutionary ecology. Evolutionary ecology, 26(5), 1187-1196.
Koricheva, J., Gurevitch, J., & Mengersen, K. (Eds.). (2013). Handbook of
meta-analysis in ecology and evolution. Princeton University Press.
Winter, S., Bauer, T., Strauss, P., Kratschmer, S., Paredes, D., Popescu,
D., ... & Zaller, J. G. (2018). Effects of vegetation management intensity
on biodiversity and ecosystem services in vineyards: A meta‐analysis.
Journal of Applied Ecology.


-- 

Giovanni Tamburini
Department of Ecology
Swedish University of Agricultural Sciences
Box 7044, SE-750 07 - Uppsala, Sweden
Tel.:+46 (0)18 672434

	[[alternative HTML version deleted]]