[R-meta] Testing interaction term of categorical moderators in rma.mv

[Viechtbauer, Wolfgang (SP)]

Please always cc the mailing list when replying.

1) There are 6 combinations, but the interaction is given by those 2 model coefficients. Please reread http://www.metafor-project.org/doku.php/tips:multiple_factors_interactions which covers exactly this case (two factors with 2 and 3 levels, respectively). So, yes, btt=5:6 tests whether there is an interaction.

2) One can mix, but it can also lead to inconsistencies. For testing fixed effects, the Wald-type test is often preferred anyway.

Best,
Wolfgang

-----Original Message-----
From: Ju Lee [mailto:juhyung2 using stanford.edu] 
Sent: Tuesday, 17 September, 2019 17:33
To: Viechtbauer, Wolfgang (SP)
Subject: Re: Testing interaction term of categorical moderators in rma.mv

Dear Wolfgang,

Thank you very much for your response.

I wanted to follow up from my previous question to clarify:

1) So my model should produce total of 6 categorical groups of interaction term (low:primary, low:secondary, low:tertiary, high:primary, high:secondary, high:tertiary). And what is shown in my mixed model below 5th and 6th coefficient is just subset of these. If I use "btt=5:6" argument, will the model still be testing the main interaction effect of "latitude x trophic level" which produces 6 interaction categories as stated above?

2) It's great to know that it is more advisable not to mix REML and ML estimation in the same study.

I deeply appreciate your time and effort, and everything you've don and are doing for this community.
I sincerely hope to hear from you!
Best wishes,
JU
________________________________________
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: Tuesday, September 17, 2019 8:00 AM
To: Ju Lee <juhyung2 using stanford.edu>; r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: RE: Testing interaction term of categorical moderators in rma.mv 
 
Hello Ju,

1) Coefficients 5 and 6 make up the interaction terms. So it would be btt=5:6. Instead of refitting the model, you can also do:

anova(r1, btt=5:6)

2) Yes, you could do that. But I would stick to the Wald-type test that is conducted with the approach above. Then you can do all your analyses with REML.

Best,
Wolfgang

-----Original Message-----
From: Ju Lee [mailto:juhyung2 using stanford.edu] 
Sent: Tuesday, 17 September, 2019 6:39
To: r-sig-meta-analysis using r-project.org
Cc: Viechtbauer, Wolfgang (SP)
Subject: Testing interaction term of categorical moderators in rma.mv

Dear Wolfgang and all, 

I am writing to ask a question of how I could perform the test to acquire main interaction effect of two categorical moderators (that have 3 and 2 levels each)
I have been referring to http://www.metafor-project.org/doku.php/tips:multiple_factors_interactions to do this, but have been troubled how to proceed.

1) In the post, Wolfgang suggests:

"To test whether the interaction is significant in general, you can either do a Wald-type test with:

rma(yi, vi, mods = ~ factor(catmod1)*factor(catmod2), data=some.data.frame, btt=X:Y)

where X is the number of the first "interaction coefficient" and Y is the number of the last "interaction coefficient" (so, these are indices to indicate which coefficients should be tested simultaneously). In the output, you will the results of this test under "Test of Moderators"." 

 However, I am not sure how I can specify the first and last coefficient of interaction terms from a output below. I would deeply appreciate if you could help me figure this out.

> r1<-rma.mv(hedged,VCV, mods=~ Region * Consumption.level, method="ML", random = ~ region.cl |   Study, data=MHF, struct="DIAG", subset=(!is.na(region.cl)))

>r1

Multivariate Meta-Analysis Model (k = 841; method: ML)

Variance Components: 

outer factor: Study     (nlvls = 176)
inner factor: region.cl (nlvls = 6)

            estim    sqrt  k.lvl  fixed                    level
tau^2.1    0.6893  0.8302    113     no    High latitude:Primary
tau^2.2    0.6354  0.7971    380     no  High latitude:Secondary
tau^2.3    1.1672  1.0804    211     no   High latitude:Tertiary
tau^2.4    1.2214  1.1052     80     no     Low latitude:Primary
tau^2.5    0.0000  0.0001     16     no   Low latitude:Secondary
tau^2.6    0.3215  0.5670     41     no    Low latitude:Tertiary

Test for Residual Heterogeneity: 
QE(df = 835) = 3679.1163, p-val < .0001

Test of Moderators (coefficient(s) 2:6): 
QM(df = 5) = 21.9742, p-val = 0.0005

Model Results:

                                               estimate      se     zval    pval    ci.lb   ci.ub    
intrcpt                                          0.3007  0.1904   1.5788  0.1144  -0.0726  0.6739    
RegionLow latitude                              -0.4096  0.3263  -1.2552  0.2094  -1.0491  0.2300    
Consumption.levelSecondary                       0.2006  0.2156   0.9303  0.3522  -0.2220  0.6232    
Consumption.levelTertiary                        0.7136  0.2497   2.8577  0.0043   0.2242  1.2031  **
RegionLow latitude:Consumption.levelSecondary    0.1687  0.3523   0.4787  0.6322  -0.5219  0.8593    
RegionLow latitude:Consumption.levelTertiary    -0.1613  0.4201  -0.3838  0.7011  -0.9847  0.6622    

---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2) Secondly, as an alternative Wolfgang suggests using likelihood-ratio test with full (with interaction term) and reduced (without interaction term) models to test the significant interaction.

However, all my models are based on "REML" method, and apparently I need to fit models with "ML" to do the above test. Is it appropriate if conduct LRT for interaction using "ML", but keep the rest of my analysis using "REML" method?

Thank you for reading, and I sincerely hope to hear back from you.

Best regards,
JU