[R-meta] Coding multi-measure correlational studies for multilevel meta-analysis

[Viechtbauer, Wolfgang (NP)]

Hi Yuhang,

First of all, I would suggest to create two separate variables for the two variables, like this:

    study    ri   ni  var1  var2
1)  1        .1   85  1     2
...
28) 1        .2   85  7     8

Then you can use rcalc() to create the var-cov matrix of the (raw or r-to-z transformed) correlation coefficients within studies (the 'V' matrix), that is, if your dataset is called 'dat', you can do:

tmp <- rcalc(ri ~ var1 + var2 | study, ni=ni, data=dat)
V <- tmp$V
dat <- tmp$dat

Sidenote: For 8 variables, there are 8*7/2 correlations (or generally, for p variables, p*(p-1)/2 -- this is one of those equations one eventually has memorized due to using it so often).

For a study (say, study 2) that used multiple measures for one of the variables (say, variable 8), there are then actually 9 variables and hence 9*8/2 = 36 correlations. The structure then is:

    study    ri   ni  var1  var2  measure1 measure2
1)  2        .1   78  1     2     a        b
...
7)  2        .3   78  1     8     a        x
8)  2        .2   78  1     8     a        y
9)  2        .4   78  2     3     b        c
...
14) 2        .3   78  2     8     b        x
15) 2        .2   78  2     8     b        y
16) 2        .5   78  3     4     c        d
...
20) 2        .4   78  3     8     c        x
21) 2        .5   78  3     8     c        y
22) 2        .1   78  4     5     d        e
...
25) 2        .0   78  4     8     d        x
26) 2        .1   78  4     8     d        y
27) 2        .3   78  5     6     e        f
...
29) 2        .3   78  5     8     e        x
30) 2        .2   78  5     8     e        y
31) 2        .2   78  6     7     f        g
32) 2        .3   78  6     8     f        x
33) 2        .1   78  6     8     f        y
34) 2        .1   78  7     8     g        x
35) 2        .2   78  7     8     g        y
36) 2        .3   78  8     8     x        y

The actual values used for measure1 and measure2 are irrelevant, as long as you use them consistently within a study. For studies that only used a single measure for each variable, you can leave measure1 and measure2 blank. For studies that used multiple measures for more than one variable, you have to keep expanding this structure. It just becomes very tedious to construct.

Then for rcalc(), you need to paste together var1 and measure1 and var2 and measure2:

dat$v1m1 <- paste0(dat$var1, ".", dat$measure1)
dat$v2m2 <- paste0(dat$var2, ".", dat$measure2)

and use those in rcalc():

tmp <- rcalc(ri ~ v1m1 + v2m2 | study, ni=ni, data=dat)
V <- tmp$V
dat <- tmp$dat

For the actual model fitted with rma.mv(), you don't use the combination of v1m1 and v2m2, but the combination of var1 and var2 as the predictor:

dat$var1var2 <- paste0(dat$var1, ".", dat$var2)

since you *want* the model not to give you estimates of 'measure-specific' pooled correlations, but you want to average over multiple measures for the same variable. So the model could be:

rma.mv(yi, V, mods = ~ 0 + var1var2, random = ~ var1var2 | study, struct="UN", data=dat)

However, this model will need to estimate 28*27/2 = 378 correlations plus 28 variances (tau^2 values) for the random effects, so in total 406 (!!) parameters (the general equation is p*(p+1)/2), plus the 28 fixed effects. That's a lot of parameters in the unstructured var-cov matrix of the random effects, so unless you have a lot of studies (hundreds if not thousands), this is going to be difficult or essentially impossible. This aside, the model allows for no heterogeneity when there are multiple correlations for the same var1var2 pair. A simple way to allow for this is to add another estimate specific random effect to the model:

dat$id <- 1:nrow(dat)
rma.mv(yi, V, mods = ~ 0 + var1var2, random = list(~ var1var2 | study, ~ 1 | id), struct="UN", data=dat)

This is simplistic, since it assumes that the heterogeneity in multiple correlations for the same pair is the same regardless of the pair. If you have a lot of data, one could try:

rma.mv(yi, V, mods = ~ 0 + var1var2, random = list(~ var1var2 | study, ~ var1var2 | id), struct=c("UN","DIAG"), data=dat)

which would use separate estimate-level random effects for each pair, but this adds another 28 parameters to the model. But who cares about another 28 if one already has 406 ...

Realistically, one needs to simplify the random effects structure. On the opposite end, there is the minimalistic:

res <- rma.mv(yi, V, mods = ~ 0 + var1var2, random = ~ 1 | study/id, data=dat)
res

which, due to its overly simplistic nature, really needs to be followed-up with:

robust(res, cluster=study, clubSandwich=TRUE)

(could do the same with the models above, but this is less likely to matter if one actually manages to fit these complex models).

An interesting question is what kind of structures of intermediate complexity one could consider.

But I'll stop here for now, since this is getting way too long anyway.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Yuhang Hu via R-sig-meta-analysis
> Sent: Wednesday, December 13, 2023 06:21
> To: R meta <r-sig-meta-analysis using r-project.org>
> Cc: Yuhang Hu <yh342 using nau.edu>
> Subject: [R-meta] Coding multi-measure correlational studies for multilevel
> meta-analysis
>
> Hello Experts,
>
> I'm collecting the correlations between 8 variables from several studies.
> If a study has used a single measure for all these 8 variables, I will need
> 28 rows (assuming no missing) to capture all those correlations i.e.,
> var1.var2 = combn(1:8, 2, FUN=\(i)paste(i,collapse = ".")):
>
>     study   ri   var1.var2
> 1)  1        .1   1.2
>  ...
> 28) 1       .2    7.8
>
> But if a study has used, say, two measures (e.g., 1, 2) for two of those 8
> variables (e.g., variables "1" and "2" in 'var1.var2'), then, I wonder how
> **best** to capture the additional 13 correlations arising due to the
> additional measure used for "1" and "2" in that study in my data for
> multilevel modeling purposes?
>
> One approach might be to add a single column called, say "measure" to add
> just those additional rows in that multi-measure study:
>
>     study   ri   var1.var2  measure
> 1)   1       .1    1.2
>  ...
> 6)   1       .6     1.7           1
> 7)   1       .4     1.7           2
> ...
> 12)  1       .8     2.7          1
> 13)  1       .7     2.7          2
> ...
>
> But this looks messy. For instance, what should be the value of "measure"
> for the var1.var2 rows that have used a single measure (e.g., var1.var2 ==
> 1.2)? And can "measure" coded this way be used in the random part of the
> model (metafor::rma.mv)?
>
> Thanks,
> Yuhang