[R-meta] "impute-the-correlation + robustness + sensitive analysis" strategy

[Célia Sofia Moreira]

Dear all,

I am performing a meta-analysis for the first time, and I would be grateful
if you could give me some recommendations / suggestions. I apologize for
this long description:

I have 20 different papers and, because some papers have more than one
experiment, 23 experiments are considered, which correspond to 23 different
samples. The complexity of my case is that each experiment includes
different domains, and each domain includes several tests. These domains
are latent variables in the individual studies (tests are the indicators).
I'm interested in studying the effects in five different domains. Thus, for
each domain, I usually collected more than one test type measure.

Finally, each sample includes an experimental and a control group. The
effects were estimated using Hedges' g (adjusted) and then calculating the
difference y between groups. In these calculations, I followed this
valuable example:
http://www.metafor-project.org/doku.php/analyses:morris2008

Unfortunately, papers do not report correlations/covariances between test
measures and, so, for each y, I only have the estimated variance v.

So, my database looks like:

ID ----- Paper --------- Sample ------- Domain ------- Test ------- y ----
v  ----  etc.

01 ------- P1  ----------  S1 ----------- D1 ---------- T1 -------- ...
02 ------- P1  ----------  S1 ----------- D1 ---------- T2 -------- ...
03 ------- P1  ----------  S1 ----------- D2 ---------- T3 -------- ...
04 ------- P1  ----------  S1 ----------- D4 ---------- T4 -------- ...
05 ------- P1  ----------  S1 ----------- D4 ---------- T5 -------- ...
06 ------- P1  ----------  S1 ----------- D4 ---------- T6 -------- ...

07 ------- P2  ----------  S2 ----------- D2 ---------- T3 -------- ...
08 ------- P2  ----------  S2 ----------- D2 ---------- T7 -------- ...
09 ------- P2  ----------  S2 ----------- D2 ---------- T8 -------- ...
10 ------- P2  ----------  S2 ----------- D3 ---------- T9 -------- ...
11 ------- P2  ----------  S2 ----------- D4 ---------- T4 -------- ...
12 ------- P2  ----------  S2 ----------- D4 ---------- T5 -------- ...
13 ------- P2  ----------  S2 ----------- D5 ---------- T0 -------- ...

14 ------- P3  ----------  S3 ----------- D1 ---------- T1 -------- ...
15 ------- P3  ----------  S4 ----------- D1 ---------- T1 -------- ...
16 ------- P3  ----------  S4 ----------- D2 ---------- T3 -------- ...

.....

I would like to assess the significance of these five domains' effects as
well as their correlation, using a meta-analyst approach. D1 is the central
domain. Thus, I thought about performing a multivariate multilevel random
meta-analysis model and then using the correlation matrix between true
effects provided in the output. Since I have no sampling covariances, I
used the "impute-the-correlation" strategy, by James Pustejovsky, together
with robust and sensitive analysis. Because I'm interested in the domains
(latent variables), the R code I'm using is the following:

Vlist <- impute_covariance_matrix(vi = dat$v, cluster = dat$Sample, r =
0.2)   # Followed by r=.4, r=.6 and r=.8
meta1 <- rma.mv(y ~ 0 + Domain, V=Vlist, random = ~ Domain | Sample, struct
= "UN", data = dat)
meta1r <- robust(meta1, cluster = dat$Sample); summary(meta1r);
r1<-cov2cor(meta1r$vb); R1 <- round(c1,2)

Does it make sense to you? Do you recommend corrections?

Introducing 2 moderators in the model, I got interesting results with this
"impute-the-correlation"+robust+sensitive strategy (ranging r from .2 to
.8), namely, the significance of D1 true effect.

However, the "impute-the-correlation" assumes equal correlations across
studies and across outcomes. I've been searching in literature, and I found
some papers reporting a high correlation between D2, D3, D4, D5: r=-6-.7
between pairs, and r=.7-.8 intra-pairs. Their correlation with D1 is lower,
and even lower for one of the pairs.

Given this different range of correlations among the outcomes, do you think
that the results I got with this strategy are truthful? Or do you think it
is better to perform the analysis in different steps, for different subsets
of variables in the multivariate multilevel model, for example, considering
two at a time, or D1 with each pair separately?

Thank you very much for your attention!

	[[alternative HTML version deleted]]