[R-meta] Does clubSandwich::coef_test() handle crossed random-effects?

[James Pustejovsky]

Hi Farzad,

clubSandwich does not support clustering in models with crossed random effects. 

There are generalizations of CRVE that work under crosses dependence structures (see Cameron, Gelbach, & Miller, 2011; Cameron & Miller, 2015). But the small-sample corrections implemented in clubSandwich have yet to be worked out for multi-way clustering. Thus, these generalizations only work when there are sufficiently large numbers of clusters *in each clustering dimension*. 

Based on currently available methods and software, I would suggest sticking with model-based inference (as implemented in rma.mv) if your interest is in a model with crossed random effects. That means you’d either have to do the work to develop a defensible V matrix and/or conduct sensitivity analysis for whatever assumptions go into constructing the V matrix.

James

Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2011). Robust inference with multiway clustering. Journal of Business & Economic Statistics, 29(2), 238-249.

Cameron, A. C., & Miller, D. L. (2015). A practitioner’s guide to cluster-robust inference. Journal of human resources, 50(2), 317-372.

> On Oct 3, 2021, at 4:20 AM, Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> 
> Hi Fred,
> 
> The dfs you see are computed based on a Satterthwaite approximation. Depending on the data structure, the dfs could very well be much lower than the number of studies/estimates.
> 
> This aside, a limitation of the cluster robust approach is that it can only capture dependencies that occur within the groups defined by the clustering variable. Or put in a different way, for the approach to work, there need to be a sufficient (for some definition of "sufficient") number of independent groups within the dataset. When introducing crossed random effects or other random effects that can create dependencies across the entire dataset (like spatial correlation or correlation due to phylogeny in meta-analyses in ecology and evolution involing multiple species), then there might only be a single (or very few) subsets in the data that are independent and then the cluster robust approach cannot really be used anymore.
> 
> Best,
> Wolfgang
> 
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>> Behalf Of Farzad Keyhan
>> Sent: Sunday, 03 October, 2021 7:35
>> To: R meta
>> Subject: [R-meta] Does clubSandwich::coef_test() handle crossed random-effects?
>> 
>> Hello Wolfgang and James,
>> 
>> I have 51 studies, but when I run an unconditional model (below),
>> coef_test() outputs a df of 4.76 (makes no sense).
>> 
>> I suspected that coef_test() is not picking up my highest-level
>> cluster (study). So, I made that explicit in the call. And then, I got
>> an error saying "random effects are not nested".
>> 
>> First, does clubSandwich handle models with crossed random-effects? If
>> not, is there an alternative code for rma.mv() models to accomplish
>> similar things (I wanted to do egger's test as well)--and of course
>> the small sample correction may not be available there.
>> 
>> Thanks, Fred
>> 
>> # Note 'scale' is crossed with 'study'
>> m1 <- rma.mv(yi, V, random = list(~1|study/es, ~1|scale), data=data)
>> 
>> coef_test(m1, "CR2")
>>   Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
>> 1 intrcpt    0.506 0.193   2.63 4.76       0.0489    *
>> 
>> ## I made cluster explicit but get an error:
>> coef_test(m1, "CR2", cluster = data$study)
>> *** Error: Random effects are not nested within clustering
> 
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]