[R-meta] Specifying the random effect structure of our multilevel meta-analysis in metafor

Viechtbauer, Wolfgang (NP) wo||g@ng@v|echtb@uer @end|ng |rom m@@@tr|chtun|ver@|ty@n|
Tue Jan 24 09:29:14 CET 2023

A few more comments from me ...


>Hi Paco,
>Responses below.
>On Mon, Jan 23, 2023 at 4:25 PM Tomas-Valiente Jorda Francisco <
>tomasf using student.ethz.ch> wrote:
>> One clarification question on the RVE approach you suggest. Your proposal
>> is that (as per the code below) we estimate our model with metafor::rma.mv
>> using some assumed variance-covariance matrix and that for hypothesis
>> testing we use clubSandwich's coef_test or coef_int or Wald_test, right?
>> Or did I misunderstand you?
>> V2 <- impute_covariance_matrix(vi=diag(V), cluster=df$experiment, r=0.6)

Don't use impute_covariance_matrix() if you have already done what you mentioned in your first post:

"Sampling errors are not independent across arms within voting propensity-experiment pairs, since the same control group (which is specific to the voting propensity-experiment pair) is used to estimate the ITT of different treatment arms. We use the Gleser & Olkin (2009) method to estimate the variance-covariance matrix of sampling errors [...]"

So you already have an appropriately calculated V matrix, so no need to make some rougher approximation of it!

>> model <- rma.mv(effect, random = list(~ 1 | experiment / voting_prop, ~ 1
>> | arm.in.experiment), data = df, V = V2, mods = ~ voting_prop)
>> coef_test(model, vcov="CR2")
>Correct. You can also accomplish the same thing using the metafor::robust()
>function, which calls clubSandwich under the hood:
>model_robust <- robust(model, cluster = experiment, clubSandwich = TRUE)
>For tests of hypotheses involving multiple parameters, you'll still need to
>use clubSandwich::Wald_test() or wildmeta::Wald_test_cwb().

The anova() function in metafor (which can be used to test multiple parameter) is actually set up to call Wald_test() appropriately (when passing a "robust.rma" object where clubSandwich was TRUE to it).

More information about the R-sig-meta-analysis mailing list