[R-meta] Random and mixed effects models with the Metafor rma.mv function

[Edwin Lebrija Trejos]

Dear Community,
I am looking for expert opinions on meta-analytical models and their implementation in The “Metafor” Package by Wolfgang Viechtbauer. 

I am checking a meta-analysis of experiments on plant species with significant implications for the reviewed topic and I am wondering about the adequacy of analyses behind some key conclusions in the study. The data of the meta-analysis consists of hundreds of observations of plant species responses taken from tens of experimental studies conducted on different species from different terrestrial plant communities and using different methodological approaches. A considerably heterogeneity in responses exist, as expected and common in ecological studies. Below I detailed three points I would appreciate to get feedback on:

1) To evaluate the "generality and magnitude" of experimental effects, the authors of the meta-analysis start by fitting a basic ‘mean’ (“random effects”) model that does not correct for any dependency on the data using the Metafor rma.mv function and the syntax: res <- rma.mv (yi, vi, data=dat), where yi are the observed effect sizes, or outcomes, and vi the corresponding sampling variances. The results of this model show a significant mean/overall effect size, as expected by theory…

Am I correct that this model, fitted with the rma.mv function, is a fixed effects model and not a random effects model (as the authors intend to fit)? 

My understanding is that when using the rma.mv function (instead of the rma.uni function), a random effects model should include a random term of the form: random = ~1| Oucome.ID, when Outcome.ID is a unique identifier for each reported experimental species response (or row in the dataset). Please clarify to me otherwise.

2) The authors emphasize that accounting for non-independence among outcomes is necessary. The focus is on the dependency of outcomes from experiments conducted on the same plant species, i.e. on a ‘taxonomic’ dependency of responses. Therefore, a “mean, corrected” model is fitted by adding a random ‘Species’ intercept to the model, i.e. res.corr <- rma.mv (yi, vi, random = list(~1|Species), data=dat). This model, as opposed to the "mean, uncorrected" model (described above), returns a weak and non-significant effect and is markedly favored by the Akaike information criterion (AIC) when compared to the "mean, corrected" model (thousands of AIC units difference). These results lead to a key conclusion that, when controlling for taxonomic non-independence in the data, there are no significant, widespread effects, as opposed to theory and generally accepted by peers. 

I am wondering as well on the formulation of such corrected model:
- Should the "corrected" model also include a random Outcome.ID term? I.e. rma.mv (yi, vi, random = list (~1|Species, ~ 1| Outcome.ID), data= dat)? 
- Moreover, agreeing that it’s important to control for dependence among outcomes, I wonder if additionally controlling for the dependence of outcomes within studies is also in place. This, since each published study used  in the meta-analysis reports experimental outcomes for several species tested in the same study. Is the following metaphor model syntax appropriate to correct for such within study dependency? rma.mv (yi, vi, random = list (~1|Species, ~1| Study.ID/ Outcome.ID), data=dat), where Study.ID is a variable that identifies each published study? 

For clarity, here is a dummy sample of the analysis data table:
Outcome.ID	Study.ID	Species 	          yi               vi
1	                 Study_1	Species A   -1.72417	0.06701
2	                 Study_1	Species A	  -1.99694	0.047748
3	                 Study_2	Species B	   0.15911	0.012989
4	                 Study_2	Species C	  -1.26529	0.115533
5	                 Study_3	Species B    0.383786	0.004959
6	                 Study_3	Species D  -0.07703	0.005961
…				

3) Follow-up analyses are conducted to explore the sources of heterogeneity in the data. These analyses are conducted by splitting the data into different categories corresponding to types of experimental methods employed, plant life stages, growth forms, climatic zones and so on. For each data subset, a “mean, corrected” model (i.e., the ‘res.corr’ model above) is fitted. I believe this what is called “Subgroup” analysis in some meta-analysis literature that I have found.  Other models fitted to further explore heterogeneity involved the inclusion of continuous variables using the ‘mods’ argument of the rma.mv function.

My question here is: isn’t it better to explore the sources of heterogenenity in the data taking advantage of the mixed model approach implemented by the rmw.mv function and include in the same model both categorical and continuous variables. Or, is there an advantage to performing” Subgroup” analysis?

Given my modest (and not too fresh) experience with meta-analysis and the Metafor package, and given the significant impact of meta-analyses on knowledge progress, I’d be very grateful if any can provide feedback and help me verify, to the extent possible, the correctness of my observations. Applying the alternative models that I mention above to the dataset used in the meta-analysis returns both quantitatively and qualitatively different results, which I find problematic.

Thanks,
Edwin