[R-meta] robust.rma.mv in metafor

[Viechtbauer, Wolfgang (NP)]

Dear Reza,

That check is related to the default way of computing the dfs (df = n-p, where n is the number of clusters and p the number of fixed effects). To see why this is a problem, try:

clubSandwich::coef_test(res, "CR2", test="naive-tp")

and you will get negative dfs here. I am cc-ing James, because clubSandwich might want to catch this. Here is a reproducible (although a bit non-sensical) example:

library(metafor)
dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, mods = ~ factor(district), random = ~ 1 | district/school, data=dat)
robust(res, cluster = district)
clubSandwich::coef_test(res, "CR2", test="naive-tp")

(here the dfs are 0, but the same issue).

clubSandwich actually uses the Satterthwaite approximation for computing the dfs by default and here the dfs are >= 1:

clubSandwich::coef_test(res, "CR2")

although they are all all quite low and so it is questionable whether those tests are trustworthy in this case.

If you still want to go ahead with this, just use coef_test(res, "CR2"). I might consider taking out the check if 'clubSandwich = TRUE' in the future, although it is still probably a sensible check because when n < p, the Satterthwaite approximation will also tend to give rather low dfs.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Reza Norouzian
>Sent: Thursday, 12 January, 2023 6:02
>To: R meta
>Subject: [R-meta] robust.rma.mv in metafor
>
>Dear Wolfgang,
>
>I'm trying to get the robust estimates of fixed effects for my model
>(res) below using robust.rma.mv().
>
>But, I wonder why I encounter an error saying:
>
>"Number of clusters (8) must be larger than the number of fixed effects (9)"
>
>Notice that when using clubSandwich directly, no errors occur.
>
>PS. Will share the data privately for reproducibility.
>
>Thank you,
>Reza Norouzian (he/him/his)
>Professor of TESOL | Anaheim University | Homepage
>Senior Researcher for Multilingual Learners | Department of Education
>###################################
>res <- rma.mv(lnr ~ outcome + 0, vi,
>       random = ~ 1|scale/study/outcome/effect,
>       data = dat)
>
>rob_res <- robust(res, cluster = scale, clubSandwich = TRUE)
>
>coef_test(res, "CR2")
>###################################