[R-meta] Constraint rrror when using Wald_test_cwb

[Viechtbauer, Wolfgang (NP)]

A follow-up question (for James) on my part.

I saw this reponse by James:

"If you only have 1 coefficient, then you don't need to go to the trouble of cluster wild bootstrapping--you can just use RVE with small sample corrections" (https://github.com/meghapsimatrix/wildmeta/issues/17#issuecomment-1833956635)

with respect to a model that includes an intercept and 1 moderator. So, if I understand this correctly, then cluster wild bootstrapping only becomes relevant (i.e., has some advantages over RVE) when the model includes more than 1 moderator?

But then I also saw this issue:

https://github.com/meghapsimatrix/wildmeta/issues/18

about the possibility to do CWB for a model with just an intercept term. So is CWB is in principle useful for this scenario?

And to answer the question in that issue: As was just discussed in another thread, it is possible to fit an rma.mv() model with only an intercept term where the coefficient is constrained to 0, with rma.mv(..., beta=0). This is undocumented, experimental, and implemented right now in a rather crude manner, but I am actually working on rma.mv() right now so that instead of profiling out the fixed effects (and then optimizing only over the variance/correlation components of the model), one can optimize over the fixed effects as well. Then one can also contrain coefficients in meta-regression models to 0. This is already possible with rma() when fitting location-scale models:

library(metafor)

dat <- dat.bcg
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat)

res1 <- rma(yi, vi, mods = ~ 1, scale = ~ 1, data=dat, method="ML", optbeta=TRUE)
res0 <- rma(yi, vi, mods = ~ ablat, scale = ~ 1, data=dat, method="ML", optbeta=TRUE, beta=c(NA,0))
res1
res0
fitstats(res0, res1)

(with method="REML", leaving out a moderator is not the same as constraining its coefficient to zero, since in REML the model matrix affects the restricted log-likelihood, but under ML as shown above, these two approaches are identical).

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Pearl, Brendan via R-sig-meta-analysis
> Sent: Sunday, September 15, 2024 01:13
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Pearl, Brendan <Brendan.Pearl using mh.org.au>
> Subject: Re: [R-meta] Constraint rrror when using Wald_test_cwb
>
> Hi James,
>
> Thankyou for this.
>
> For anyone else who stumbles on this question, I also found that James answered
> it here: https://github.com/meghapsimatrix/wildmeta/issues/17#issuecomment-
> 1833956635
>
> ________________________________
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> on behalf
> of James Pustejovsky via R-sig-meta-analysis <r-sig-meta-analysis using r-project.org>
> Sent: Saturday, 14 September 2024 10:54:55 PM
> To: R Special Interest Group for Meta-Analysis
> Cc: James Pustejovsky
> Subject: Re: [R-meta] Constraint rrror when using Wald_test_cwb
>
> Wald_test_cwb() is for testing null hypotheses specified by a constraint or
> constraints on the model coefficients. In your MWE, you fit a summary meta-
> analysis with only one beta coefficient, so the constraints can only refer to
> that first coefficient (hence “Constraint indices must be less than or equal to
> 1”).
>
> As an aside, note that the same error would occur if you use the more basic
> Wald_test() from clubSandwich.
>
> James
>
> > On Sep 14, 2024, at 7:32 AM, Michael Dewey via R-sig-meta-analysis <r-sig-
> meta-analysis using r-project.org> wrote:
> >
> > Dear Brendan
> >
> > When I run your MWE (after inserting library(metafor) I get
> >
> > Error in Wald_test_cwb(full_model = meta_analysis_robust, constraints =
> constrain_equal(1:3),  :
> >  could not find function "Wald_test_cwb"
> >
> > Michael
> >
> >> On 14/09/2024 10:28, Pearl, Brendan via R-sig-meta-analysis wrote:
> >> Hello,
> >> I am trying to run a cluster wild bootstrap, but am getting the following
> error:
> >> Error in constrain_zero(constraints = constraints, coefs = coefs) :
> >>   Constraint indices must be less than or equal to 1.
> >>   Question: What does this mean?
> >> Thankyou,
> >> Brendan
> >> Background (if relevant):
> >> I am running a purely exploratory series of meta-analyses of the
> relationships between several predictors and outcomes (i.e. n x m meta-
> analyses).
> >> There is non-independence within each predictor-relationship pair (some
> studies report multiple effect sizes for the same group of participants) and the
> effect sizes are nested.
> >> I am following the general workflow outlined here:
> (https://wviechtb.github.io/metafor/reference/misc-recs.html#general-workflow-
> for-meta-analyses-involving-complex-dependency-structures) and want to use
> cluster wild bootstrapping because some analyses have very few studies (and
> cluster-robust inference methods led to very wide confidence intervals)
> >> Minimal working example:
> >> ```{r}
> >> dat_temp_mwe <- structure(list(study = c("A", "B", "C", "D", "E", "E", "F",
> "F",
> >>     "F", "F", "G"), effect_id = c(11, 28, 73, 93, 115, 232, 236,
> >>     242, 252, 266, 284), Paper = c("AA", "BB", "CC", "DD", "EE1",
> >>     "EE2", "FF1", "FF2", "FF3", "FF4", "GG"), Mean_age_when_outcome_measured
> = c(21,
> >>     19, 26, 19, 19, 21, 21, 21, 21, 19, 19), yi = structure(c(-
> 0.0401817896328312,
> >>     -0.0700000000000002, -0.151002873536528, -0.113328685307003,
> >>     -0.139761942375159, -0.0392207131532808, -0.0487901641694324,
> >>     -0.05, -0.041141943331175, -0.0421011760186351, -0.604315966853329
> >>     ), ni = c(1566, 844, 624, 355, 7449, 2135, 2410, 4853, 6912,
> >>     7842, 1202), measure = "GEN"), vi = c(0.00014647659162424,
> 0.000527143487544687,
> >>     0.00336452354486442, 0.00116040765035603, 0.00667694039383453,
> >>     9.6295107522168e-05, 9.44692075770055e-05, 0.000100003675148229,
> >>     2.50009187870589e-05, 2.50009187870581e-05, 0.0292124283937479
> >>     )), row.names = c(NA, 11L), class = c("escalc", "data.frame"), yi.names =
> "yi", vi.names = "vi", digits = c(est = 4,
> >>     se = 4, test = 4, pval = 4, ci = 4, var = 4, sevar = 4, fit = 4,
> >>     het = 4))
> >>     V <- vcalc(vi,
> >>     cluster = study,
> >>     obs = effect_id,
> >>     time1 = Mean_age_when_outcome_measured,
> >>     data = dat_temp_mwe,
> >>     rho = 0.8,
> >>     phi = 0.9)
> >>   meta_analysis_output <- rma.mv(
> >>     yi,
> >>     V = V,
> >>     random = ~ 1 | study / Paper / effect_id,
> >>     data = dat_temp_mwe,
> >>     control = list(rel.tol = 1e-8))
> >>   Wald_test_cwb(full_model = meta_analysis_robust,
> >>     constraints = constrain_equal(1:3),
> >>     R = 99,
> >>     seed = 20201229)
> >> ```