[R-meta] Testing of moderators in rma()

[James Pustejovsky]

When the model includes an intercept term, the omnibus test does *not*
include the intercept. So the null hypothesis is b1 = 0 and b2 =0.

If you fit the model without the intercept, then the equivalent to the
omnibus test from the model with an intercept would be that the average
effect sizes are all equal, as in b1 = b2 = b3.

On Tue, Oct 24, 2017 at 4:29 PM, Samuel Knapp <samuel.knapp at tum.de> wrote:

> Many thanks, that made it very clear.
>
> For the first case, would the null hypothesis not have to be b0=b1=b2?
> This is how I understood your explanation.
>
> To me, the eqn (eg. b1=0) look very similar between both examples.
>
> Best regards
>
> On 24/10/17 23:21, James Pustejovsky wrote:
>
> Samuel,
>
> These two omnibus tests have very different interpretations. Say that you
> have a moderator with three levels, A, B, C.
>
> If you fit the model with an intercept, as in
> yi = b0 + b1 X1i + b2 X2i + ei,
> then b0 will be the average effect size for the reference level (say this
> is A), b1 will be the difference in average effect sizes comparing B versus
> A, and b2 will be the difference in average effect sizes comparing C versus
> A. The omnibus test in this case is for the null hypothesis that b1 = 0 and
> b2 = 0, i.e., that the average effect sizes are all equal to b0*. *Note
> too that this omnibus test would have 2 degrees of freedom.
>
> If you fit the model without an intercept, as in
> yi = b1 X1i + b2 X2i + b3 X3i + ei,
> then b1 will be the average effect size for studies with level A, b2 will
> be the average effect size for studies with level B, and b3 will be the
> average effect size for studies with level C. The omnibus test in this case
> is for the null hypothesis that b1 = 0, b2 = 0, and b3 = 0, i.e., that the
> average effect sizes are all equal to zero. Note too that this omnibus test
> would have 3 degrees of freedom.
>
> So the two omnibus tests are quite different, and there is no reason to
> expect that they should be consistent with each other.
>
> James
>
> On Tue, Oct 24, 2017 at 3:55 PM, Samuel Knapp <samuel.knapp at tum.de> wrote:
>
>> Dear all,
>>
>> I have a problem in finding the right test for the inclusion of
>> moderators, or actually I'm not sure if I should include the intercept term
>> or not. What troubles me, is that the removal of the intercept term, has a
>> very big effect on the omnibus test of the moderators.
>>
>> The model: rma.mv() with an additional random effect (study), a
>> variance-covariance matrix for the sampling variances and covariances
>> (Lajeunesse correction).
>>
>> I want to test species as a moderator. When I include the intercept, the
>> moderator effect is not significant (P=0.2779), and when I remove the
>> intercept P<0.001. I started to remove the intercept to get the average
>> effects for levels for each species and the z-test for each species.
>> However, no I'm not sure anymore, what the different interpretation of
>> moderator test for the two different models are.
>>
>> Thanks a lot!
>>
>> ### Model with intercept:
>>
>> > specmodel <- rma.mv(yi~species,V=varmat,ran
>> dom=~1|study/myo,data=metadat,method="REML")
>> > summary(specmodel)
>>
>> Multivariate Meta-Analysis Model (k = 166; method: REML)
>>
>>   logLik  Deviance       AIC       BIC      AICc
>>  12.8545  -25.7089   22.2911   93.5666   32.3751
>>
>> Variance Components:
>>
>>             estim    sqrt  nlvls  fixed     factor
>> sigma^2.1  0.0216  0.1470     39     no      study
>> sigma^2.2  0.0300  0.1732    166     no  study/myo
>>
>> Test for Residual Heterogeneity:
>> QE(df = 144) = 1386.5618, p-val < .0001
>>
>> Test of Moderators (coefficient(s) 2:22):
>> QM(df = 21) = 24.3187, p-val = 0.2779
>>
>> ### Model without intercept:
>>
>> > specmodel <- rma.mv(yi~species-1,V=varmat,r
>> andom=~1|study/myo,data=metadat,method="REML")
>> > summary(specmodel)
>>
>> Multivariate Meta-Analysis Model (k = 166; method: REML)
>>
>>   logLik  Deviance       AIC       BIC      AICc
>>  12.8545  -25.7089   22.2911   93.5666   32.3751
>>
>> Variance Components:
>>
>>             estim    sqrt  nlvls  fixed     factor
>> sigma^2.1  0.0216  0.1470     39     no      study
>> sigma^2.2  0.0300  0.1732    166     no  study/myo
>>
>> Test for Residual Heterogeneity:
>> QE(df = 144) = 1386.5618, p-val < .0001
>>
>> Test of Moderators (coefficient(s) 1:22):
>> QM(df = 22) = 61.9539, p-val < .0001
>>
>>
>> --
>> Samuel Knapp
>>
>> Lehrstuhl für Pflanzenernährung
>> Technische Universität München
>> (Chair of Plant Nutrition
>> Technical University of Munich)
>>
>> Emil-Ramann-Strasse 2
>> D-85354 Freising
>>
>> Tel. +49 8161 71-3578
>> samuel.knapp at tum.de
>> www.researchgate.net/profile/Samuel_Knapp
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
>
>
>

	[[alternative HTML version deleted]]