[R-meta] "impute-the-correlation + robustness + sensitive analysis" strategy

[James Pustejovsky]

Celia,

If you have reasonable prior information about the correlation between
certain tests, which suggests that some tests are more highly correlated
then others, then I would definitely recommend using that information.

In principle, you can still use the "impute-the-correlation" strategy even
while assuming unequal correlations between certain tests. For instance, as
you suggested, you might assume r = 0.7 for the inter-correlation between
T1 through T4, but then assume r = 0.2 for the correlation between these
measures and T5. For sensitivity analysis, you could vary these
correlations by adding/subtracting 0.1 or 0.2 from each (off-diagonal)
cell.

Another useful sensitivity analysis would be to assume zero correlations
between tests from different domains and then also use the struct = "DIAG"
argument in rma.mv. This amounts to estimating separate (marginal) models
for each domain. If you get very different average effect estimates than
you do with the full multivariate model, then it would indicate that the
results are going to be fairly sensitive to the assumption you make about
the cross-domain correlations.

Of course, implementing these approaches takes a bit more work. One
approach would be to create the variance-covariance matrices for each study
"by hand," and then store them in a list that can be fed into rma.mv. This
is tedious but might it be the easiest way to go. I have an idea for how to
make the impute_covariance_matrix() function more helpful for your
use-case, but it will be a week or two before I can get to it.

James



On Tue, Jan 23, 2018 at 6:16 AM, Célia Sofia Moreira <
celiasofiamoreira at gmail.com> wrote:

> Dear all,
>
> I am performing a meta-analysis for the first time, and I would be grateful
> if you could give me some recommendations / suggestions. I apologize for
> this long description:
>
> I have 20 different papers and, because some papers have more than one
> experiment, 23 experiments are considered, which correspond to 23 different
> samples. The complexity of my case is that each experiment includes
> different domains, and each domain includes several tests. These domains
> are latent variables in the individual studies (tests are the indicators).
> I'm interested in studying the effects in five different domains. Thus, for
> each domain, I usually collected more than one test type measure.
>
> Finally, each sample includes an experimental and a control group. The
> effects were estimated using Hedges' g (adjusted) and then calculating the
> difference y between groups. In these calculations, I followed this
> valuable example:
> http://www.metafor-project.org/doku.php/analyses:morris2008
>
> Unfortunately, papers do not report correlations/covariances between test
> measures and, so, for each y, I only have the estimated variance v.
>
> So, my database looks like:
>
> ID ----- Paper --------- Sample ------- Domain ------- Test ------- y ----
> v  ----  etc.
>
> 01 ------- P1  ----------  S1 ----------- D1 ---------- T1 -------- ...
> 02 ------- P1  ----------  S1 ----------- D1 ---------- T2 -------- ...
> 03 ------- P1  ----------  S1 ----------- D2 ---------- T3 -------- ...
> 04 ------- P1  ----------  S1 ----------- D4 ---------- T4 -------- ...
> 05 ------- P1  ----------  S1 ----------- D4 ---------- T5 -------- ...
> 06 ------- P1  ----------  S1 ----------- D4 ---------- T6 -------- ...
>
> 07 ------- P2  ----------  S2 ----------- D2 ---------- T3 -------- ...
> 08 ------- P2  ----------  S2 ----------- D2 ---------- T7 -------- ...
> 09 ------- P2  ----------  S2 ----------- D2 ---------- T8 -------- ...
> 10 ------- P2  ----------  S2 ----------- D3 ---------- T9 -------- ...
> 11 ------- P2  ----------  S2 ----------- D4 ---------- T4 -------- ...
> 12 ------- P2  ----------  S2 ----------- D4 ---------- T5 -------- ...
> 13 ------- P2  ----------  S2 ----------- D5 ---------- T0 -------- ...
>
> 14 ------- P3  ----------  S3 ----------- D1 ---------- T1 -------- ...
> 15 ------- P3  ----------  S4 ----------- D1 ---------- T1 -------- ...
> 16 ------- P3  ----------  S4 ----------- D2 ---------- T3 -------- ...
>
> .....
>
> I would like to assess the significance of these five domains' effects as
> well as their correlation, using a meta-analyst approach. D1 is the central
> domain. Thus, I thought about performing a multivariate multilevel random
> meta-analysis model and then using the correlation matrix between true
> effects provided in the output. Since I have no sampling covariances, I
> used the "impute-the-correlation" strategy, by James Pustejovsky, together
> with robust and sensitive analysis. Because I'm interested in the domains
> (latent variables), the R code I'm using is the following:
>
> Vlist <- impute_covariance_matrix(vi = dat$v, cluster = dat$Sample, r =
> 0.2)   # Followed by r=.4, r=.6 and r=.8
> meta1 <- rma.mv(y ~ 0 + Domain, V=Vlist, random = ~ Domain | Sample,
> struct
> = "UN", data = dat)
> meta1r <- robust(meta1, cluster = dat$Sample); summary(meta1r);
> r1<-cov2cor(meta1r$vb); R1 <- round(c1,2)
>
> Does it make sense to you? Do you recommend corrections?
>
> Introducing 2 moderators in the model, I got interesting results with this
> "impute-the-correlation"+robust+sensitive strategy (ranging r from .2 to
> .8), namely, the significance of D1 true effect.
>
> However, the "impute-the-correlation" assumes equal correlations across
> studies and across outcomes. I've been searching in literature, and I found
> some papers reporting a high correlation between D2, D3, D4, D5: r=-6-.7
> between pairs, and r=.7-.8 intra-pairs. Their correlation with D1 is lower,
> and even lower for one of the pairs.
>
> Given this different range of correlations among the outcomes, do you think
> that the results I got with this strategy are truthful? Or do you think it
> is better to perform the analysis in different steps, for different subsets
> of variables in the multivariate multilevel model, for example, considering
> two at a time, or D1 with each pair separately?
>
> Thank you very much for your attention!
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]