[R-meta] Should I fit both the only-intercept and the meta-regression?

[Gabriele Midolo]

 Hi Luca,

In my opinion, and based on what you described, this is what the estimate
of model B seems to suggest (i.e. that the estimated effect size is not
significantly different from 0 when the moderator has a low value). I
assume the moderator you are using is a continuous variable, but to
visualize/address it differently, you might also subset the variable to
more levels and then build a  so-called forest plot to see how the
significance of pooled effect size changes along the levels. There is a
nice example of this in an ecological meta-analyisis conducted by
Shulte-Uebbing and de Vries (2017) (Figure 2) (
https://doi.org/10.1111/gcb.13862).
Hope you will get extra comments from other people too.

Good luck with your work!
Gabriele

On 20 March 2018 at 17:31, Luca Santini <luca.santini.eco at gmail.com> wrote:

> Hi Gabriele,
>
> Thanks a lot for sharing your thoughts. This is useful and makes sense to
> me. My objective is two-folds: test whether the summary effect size is
> different from 0 (and goes in the expected direction) and test whether the
> variability in the effect size could arise from the effect of a variable
> (my moderator). So probably the two-step approach is what I need.
> However I am still unclear about the interpretation of my metaregression
> result. If the intercept of model B becomes significant only when the
> moderator is included, can I conclude the hypothesis I’m testing with the
> only-intercept model is supported only under certain conditions (low values
> of the moderator variable)?
> Thanks
>
> Luca
>
> On 20 Mar 2018, at 11:24, Gabriele Midolo <gabriele.midolo at gmail.com>
> wrote:
>
> Hi Luca,
>
> I am not properly an expert on metafor too, so better hear from the others
> as well!
>
> I think the answer to your question depends on what you want to test? What
> I do for my meta-analyses, I run a null-model (without moderators) via rma
> or rma.mv (depending on how data are structured) to obtain a mean pooled
> estimate of the effect size (i.e. the intercept of the model) which is
> something I would report first in the results of a paper. Then, a second
> step is to look at moderators in meta-regression as you said, and I guess
> you can interpret these as linear mixed effect models. So, the intercept in
> this case tells you the estimated effect size when your moderator is = 0. I
> guess using AIC/BIC/et cetera is a good way to select models when you have
> multiple moderators. In general, you might also be interested to look at
> "omnibus test of moderators" statistic provided in metafor which assess the
> importance of moderatos in a model and if they significanlty contribute to
> reduce heterogeneity?.
>
> I don't know if this help and hope people agree with what I've put above!
>
> Gabriele
>
> On 20 March 2018 at 08:28, Luca Santini <luca.santini.eco at gmail.com>
> wrote:
>
>> Dear all,
>>
>> I’m totally new to meta-analytical approaches, I am running my first
>> meta-analysis using "metafor" but I am now a bit confused about the
>> interpretation.
>>
>> I am running 2 mixed-effect meta-analyses (only-intercept models) on
>> correlation coefficients (models A and B). I have an hypothesis for a
>> possible continuous moderator for both of the mixed-effect models.
>> When I run the meta-analyses without the moderator, the intercept of
>> model A is significant and supports the hypothesis. When I run the
>> meta-analyses with the moderator (metaregressions), both the intercept and
>> the moderator coefficients of model B are significant, whereas neither of
>> the coefficients of model A are significant.
>> In other words, my conclusions change if I use the simple mixed-effect
>> meta-analysis, or the mixed-effect meta-analyis with the moderator
>> (metaregression).
>>
>> So my question is,
>> - should I run both of them and use only-intercept model to test my
>> hypothesis and the metaregression just as a test for the moderator variable?
>> OR
>> - should I only run the metaregression and interpret both the intercept
>> and the moderator coefficient?
>> OR
>> - should I run the meta-regression directly but only retain moderators if
>> supported by information criteria (AIC, BIC, DIC or similar?)
>>
>> Note that if I apply the third option using AIC, the moderator of model A
>> is excluded whereas the moderator of model B is retained, resulting in both
>> models being significant.
>>
>> I found both cases in the literature, authors that use the simple
>> meta-analysis and separately the metaregression to test the effect of
>> moderators and authors that only run the metaregression, so I’m a bit
>> confused.
>>
>> Would you be so kind to share your expert opinion on this? Thanks in
>> advance.
>>
>> Best regards
>>
>> Luca
>>
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>
>
>

	[[alternative HTML version deleted]]