[R-meta] Testing of moderators in rma()

[James Pustejovsky]

Samuel,

These two omnibus tests have very different interpretations. Say that you
have a moderator with three levels, A, B, C.

If you fit the model with an intercept, as in
yi = b0 + b1 X1i + b2 X2i + ei,
then b0 will be the average effect size for the reference level (say this
is A), b1 will be the difference in average effect sizes comparing B versus
A, and b2 will be the difference in average effect sizes comparing C versus
A. The omnibus test in this case is for the null hypothesis that b1 = 0 and
b2 = 0, i.e., that the average effect sizes are all equal to b0*. *Note too
that this omnibus test would have 2 degrees of freedom.

If you fit the model without an intercept, as in
yi = b1 X1i + b2 X2i + b3 X3i + ei,
then b1 will be the average effect size for studies with level A, b2 will
be the average effect size for studies with level B, and b3 will be the
average effect size for studies with level C. The omnibus test in this case
is for the null hypothesis that b1 = 0, b2 = 0, and b3 = 0, i.e., that the
average effect sizes are all equal to zero. Note too that this omnibus test
would have 3 degrees of freedom.

So the two omnibus tests are quite different, and there is no reason to
expect that they should be consistent with each other.

James

On Tue, Oct 24, 2017 at 3:55 PM, Samuel Knapp <samuel.knapp at tum.de> wrote:

> Dear all,
>
> I have a problem in finding the right test for the inclusion of
> moderators, or actually I'm not sure if I should include the intercept term
> or not. What troubles me, is that the removal of the intercept term, has a
> very big effect on the omnibus test of the moderators.
>
> The model: rma.mv() with an additional random effect (study), a
> variance-covariance matrix for the sampling variances and covariances
> (Lajeunesse correction).
>
> I want to test species as a moderator. When I include the intercept, the
> moderator effect is not significant (P=0.2779), and when I remove the
> intercept P<0.001. I started to remove the intercept to get the average
> effects for levels for each species and the z-test for each species.
> However, no I'm not sure anymore, what the different interpretation of
> moderator test for the two different models are.
>
> Thanks a lot!
>
> ### Model with intercept:
>
> > specmodel <- rma.mv(yi~species,V=varmat,ran
> dom=~1|study/myo,data=metadat,method="REML")
> > summary(specmodel)
>
> Multivariate Meta-Analysis Model (k = 166; method: REML)
>
>   logLik  Deviance       AIC       BIC      AICc
>  12.8545  -25.7089   22.2911   93.5666   32.3751
>
> Variance Components:
>
>             estim    sqrt  nlvls  fixed     factor
> sigma^2.1  0.0216  0.1470     39     no      study
> sigma^2.2  0.0300  0.1732    166     no  study/myo
>
> Test for Residual Heterogeneity:
> QE(df = 144) = 1386.5618, p-val < .0001
>
> Test of Moderators (coefficient(s) 2:22):
> QM(df = 21) = 24.3187, p-val = 0.2779
>
> ### Model without intercept:
>
> > specmodel <- rma.mv(yi~species-1,V=varmat,r
> andom=~1|study/myo,data=metadat,method="REML")
> > summary(specmodel)
>
> Multivariate Meta-Analysis Model (k = 166; method: REML)
>
>   logLik  Deviance       AIC       BIC      AICc
>  12.8545  -25.7089   22.2911   93.5666   32.3751
>
> Variance Components:
>
>             estim    sqrt  nlvls  fixed     factor
> sigma^2.1  0.0216  0.1470     39     no      study
> sigma^2.2  0.0300  0.1732    166     no  study/myo
>
> Test for Residual Heterogeneity:
> QE(df = 144) = 1386.5618, p-val < .0001
>
> Test of Moderators (coefficient(s) 1:22):
> QM(df = 22) = 61.9539, p-val < .0001
>
>
> --
> Samuel Knapp
>
> Lehrstuhl für Pflanzenernährung
> Technische Universität München
> (Chair of Plant Nutrition
> Technical University of Munich)
>
> Emil-Ramann-Strasse 2
> D-85354 Freising
>
> Tel. +49 8161 71-3578
> samuel.knapp at tum.de
> www.researchgate.net/profile/Samuel_Knapp
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]