[R-meta] Potential bug in clubSandwich::Wald_test()

[Stefanou Revesz]

Thank you very much, James.  Does changing the `test=` to another option
(e.g., "Naive-F") but keeping the `vcov="CR2"` might still be better than
the QM provided by the rma.mv() model itself?

Wald_test(fit, constrain_zero(fit$btt), "CR2", test = "Naive-F")

Stefanou

On Tue, Mar 1, 2022 at 2:49 PM James Pustejovsky <jepusto using gmail.com> wrote:

> I cannot reproduce the error. Here is what I get:
> > Wald_test(fit, constrain_zero(fit$btt), "CR2")
>  test Fstat df_num df_denom p_val sig
>   HTZ     0     15    -0.55    NA
>
> The test you are trying to run is for a configuration where the HTZ test
> does not work well. It is a joint test of 15 coefficients but the data
> include only 28 distinct studies, so this is a very challenging scenario in
> terms of getting the small-sample approximation correct. In particular, the
> approximation leads to negative degrees of freedom and a corrected F
> statistic of zero because the number of coefficients is so large and the
> number of clusters is limited. In this situation, I would suggest using a
> cluster wild bootstrap test instead, as described in a recent paper by
> Megha Joshi and me:
>
> https://www.jepusto.com/publication/cluster-wild-bootstrap-for-meta-analysis/
>
> The syntax requires only minor modifications (although it takes a bit
> longer to compute):
>
> > library(wildmeta)
> > Wald_test_cwb(fit, constraints = constrain_zero(fit$btt), R = 1999, seed
> = 20220301)
>   Test Adjustment CR_type Statistic    R      p_val
> 1  CWB        CR0     CR0   Naive-F 1999 0.03951976
>
> James
>
> On Tue, Mar 1, 2022 at 11:15 AM Stefanou Revesz <stefanourevesz using gmail.com>
> wrote:
>
>> Dear Dr. Pustejovsky (and other Meta Experts),
>>
>> I've run into a potential bug in clubSandwich::Wald_test().
>>
>> Would you please confirm whether the error in the following reproducible
>> example is a bug or not? Any possible fix?
>>
>> Best,
>> Stefanou
>>
>> d <- read.csv("https://raw.githubusercontent.com/fpqq/w/main/f.csv")
>>
>> fit <- rma.mv(gi ~ teaching_level +pd_format +pd_content +
>>                       training_hr +time +outcome +area +
>>                        time_wk_whn +time_wk_btw +study_type,
>>              V = v_gi,
>>              random = ~1|study/obs,
>>              data = d)
>>
>> Wald_test(fit, constrain_zero(fit$btt), "CR2")
>>
>> Error in symnum(x$p_val, corr = FALSE, na = FALSE, cutpoints = c(0,
>> 0.001,  :
>>   must have 2 'symbols' for logical 'x' argument
>>
>

	[[alternative HTML version deleted]]