[R-meta] Cross-Classified Random-Effects Model in rma.mv

[Viechtbauer, Wolfgang (SP)]

Dear Mark,

Indeed,

rma.mv(yi, vi, random = list(~ 1 | study/effectsize, ~ 1 | instrument), data=data)

will be a model with crossed random effects (that is, 'instrument' is not nested, but a crossed random effect).

Whether 'instrument' should be considered nested within studies or treated as a crossed random effect is debatable. Actually, unless the same instrument was used multiple times within at least some of the studies (e.g., study 4 provides three effect sizes, *two* with instrument 1 and one with instrument 2), the model

rma.mv(yi, vi, random = ~ 1 | study/instrument/effectsize, data=data)

is overparameterized (since the instrument-level heterogeneity cannot be distinguished from the effectsize-level heterogeneity).

I haven't read the article by Fernández-Castilla et al. (2018) -- but thanks for bringing it to my attention! -- so I cannot tell you what they propose in their appendices. But hopefully the above is still useful to you (at least I can confirm that the syntax is correct).

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Assink, Mark
Sent: Friday, 18 January, 2019 16:49
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] Cross-Classified Random-Effects Model in rma.mv

Dear Wolfgang and other members,

In a recent paper of Fern�ndez-Castilla and colleagues (2018; https://doi.org/10.3758/s13428-018-1063-2), it is explained how a cross-classified random effects model (CCREM) can be fitted in SAS. I was wondering whether and how CCREM's can be fitted in R using the rma.mv function of the metafor package.

Following the above cited paper, suppose you have a meta-analytic structure in which effect sizes are nested within studies:

* Level 1 -> Variability in effect sizes due to sampling variance;
* Level 2 -> Variability in effect sizes extracted from the same studies (i.e., within-study variance);
* Level 3 -> Variability in effect sizes extracted from different studies (i.e., between-study variance).

Let's say that across primary studies multiple/different instruments were used to measure a specific outcome. For example, three effect sizes from study 1 were based on instruments 1, 2, and 3; two effect sizes from study 2 were based on instruments 1 and 4; three effect sizes from study 3 were based on instruments 2, 4, and 5, etc. So, the variable "instrument" can be regarded as a crossed factor.

To model the above structure using the rma.mv function, I would write:

rma.mv(yi, vi, random = list(~ 1 | study/effectsize, ~ 1 | instrument), data=data)

I assume that with this syntax, the clustering or dependency of effect sizes within studies is accounted for, while the variation in effect sizes based on the different instruments that are used (between-instrument variance) is also modeled.

However, I am not sure whether this would be correct as Fern�ndez-Castilla et al. (2018) refer to "random factors nested within studies" in their appendices with SAS codes. I'd say that a variable like "instrument" from the example above would not be nested within studies, because the same instrument(s) are used across studies.

Are my reasoning and R-syntax correct? I highly appreciate any reflection, help, or suggestion.

Best,
Mark