[R-meta] Should I fit both the only-intercept and the meta-regression?

[Luca Santini]

Hi Gabriele,

Thanks a lot for sharing your thoughts. This is useful and makes sense to me. My objective is two-folds: test whether the summary effect size is different from 0 (and goes in the expected direction) and test whether the variability in the effect size could arise from the effect of a variable (my moderator). So probably the two-step approach is what I need.
However I am still unclear about the interpretation of my metaregression result. If the intercept of model B becomes significant only when the moderator is included, can I conclude the hypothesis I’m testing with the only-intercept model is supported only under certain conditions (low values of the moderator variable)? 
Thanks

Luca

> On 20 Mar 2018, at 11:24, Gabriele Midolo <gabriele.midolo at gmail.com> wrote:
> 
> Hi Luca,
> 
> I am not properly an expert on metafor too, so better hear from the others as well!
> 
> I think the answer to your question depends on what you want to test? What I do for my meta-analyses, I run a null-model (without moderators) via rma or rma.mv <http://rma.mv/> (depending on how data are structured) to obtain a mean pooled estimate of the effect size (i.e. the intercept of the model) which is something I would report first in the results of a paper. Then, a second step is to look at moderators in meta-regression as you said, and I guess you can interpret these as linear mixed effect models. So, the intercept in this case tells you the estimated effect size when your moderator is = 0. I guess using AIC/BIC/et cetera is a good way to select models when you have multiple moderators. In general, you might also be interested to look at "omnibus test of moderators" statistic provided in metafor which assess the importance of moderatos in a model and if they significanlty contribute to reduce heterogeneity?.
> 
> I don't know if this help and hope people agree with what I've put above!
> 
> Gabriele
> 
> On 20 March 2018 at 08:28, Luca Santini <luca.santini.eco at gmail.com <mailto:luca.santini.eco at gmail.com>> wrote:
> Dear all,
> 
> I’m totally new to meta-analytical approaches, I am running my first meta-analysis using "metafor" but I am now a bit confused about the interpretation.
> 
> I am running 2 mixed-effect meta-analyses (only-intercept models) on correlation coefficients (models A and B). I have an hypothesis for a possible continuous moderator for both of the mixed-effect models.
> When I run the meta-analyses without the moderator, the intercept of model A is significant and supports the hypothesis. When I run the meta-analyses with the moderator (metaregressions), both the intercept and the moderator coefficients of model B are significant, whereas neither of the coefficients of model A are significant.
> In other words, my conclusions change if I use the simple mixed-effect meta-analysis, or the mixed-effect meta-analyis with the moderator (metaregression).
> 
> So my question is,
> - should I run both of them and use only-intercept model to test my hypothesis and the metaregression just as a test for the moderator variable?
> OR
> - should I only run the metaregression and interpret both the intercept and the moderator coefficient?
> OR
> - should I run the meta-regression directly but only retain moderators if supported by information criteria (AIC, BIC, DIC or similar?)
> 
> Note that if I apply the third option using AIC, the moderator of model A is excluded whereas the moderator of model B is retained, resulting in both models being significant.
> 
> I found both cases in the literature, authors that use the simple meta-analysis and separately the metaregression to test the effect of moderators and authors that only run the metaregression, so I’m a bit confused.
> 
> Would you be so kind to share your expert opinion on this? Thanks in advance.
> 
> Best regards
> 
> Luca
> 
> 
>         [[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org <mailto:R-sig-meta-analysis at r-project.org>
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis <https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis>
> 


	[[alternative HTML version deleted]]