[R-meta] Specifying the random effect structure of our multilevel meta-analysis in metafor

[Viechtbauer, Wolfgang (NP)]

A few more comments from me ...

Best,
Wolfgang

>Hi Paco,
>
>Responses below.
>
>James
>
>On Mon, Jan 23, 2023 at 4:25 PM Tomas-Valiente Jorda Francisco <
>tomasf using student.ethz.ch> wrote:
>
>> One clarification question on the RVE approach you suggest. Your proposal
>> is that (as per the code below) we estimate our model with metafor::rma.mv
>> using some assumed variance-covariance matrix and that for hypothesis
>> testing we use clubSandwich's coef_test or coef_int or Wald_test, right?
>> Or did I misunderstand you?
>>
>> V2 <- impute_covariance_matrix(vi=diag(V), cluster=df$experiment, r=0.6)

Don't use impute_covariance_matrix() if you have already done what you mentioned in your first post:

"Sampling errors are not independent across arms within voting propensity-experiment pairs, since the same control group (which is specific to the voting propensity-experiment pair) is used to estimate the ITT of different treatment arms. We use the Gleser & Olkin (2009) method to estimate the variance-covariance matrix of sampling errors [...]"

So you already have an appropriately calculated V matrix, so no need to make some rougher approximation of it!

>> model <- rma.mv(effect, random = list(~ 1 | experiment / voting_prop, ~ 1
>> | arm.in.experiment), data = df, V = V2, mods = ~ voting_prop)
>>
>> coef_test(model, vcov="CR2")
>>
>Correct. You can also accomplish the same thing using the metafor::robust()
>function, which calls clubSandwich under the hood:
>
>model_robust <- robust(model, cluster = experiment, clubSandwich = TRUE)
>
>For tests of hypotheses involving multiple parameters, you'll still need to
>use clubSandwich::Wald_test() or wildmeta::Wald_test_cwb().

The anova() function in metafor (which can be used to test multiple parameter) is actually set up to call Wald_test() appropriately (when passing a "robust.rma" object where clubSandwich was TRUE to it).