[R-meta] "impute-the-correlation + robustness + sensitive analysis" strategy

[Célia Sofia Moreira]

Dear James,

Thank you very much for your quick reply.

You write that "assume r = 0.7 for the inter-correlation between T1 through
T4, but then assume r = 0.2 for the correlation between these measures and
T5." However, I can not assume two different correlations simultaneously
with the impute_the_covariance function. So, I presume you are suggesting
to run the "impute_the_covariance + rma.mv" successively for r=.2,..., r=.7
(steps of .1 or .2), isn't is? In affirmative case, is the R code I wrote
- in my previous message - the adequate?

About creating the variance-covariance matrix by hand is a very complex
procedure because correlations across domains vary from paper to paper, and
I would have to consider a reasonable number of matrices resulting from
different correlations combinations.

I would be grateful if you could provide me a generalization of the
impute_the_covariance function to my use-case. I will definitely wait.
Thank you! Please let me know when it's ready :)

Kind regards,
 celia

2018-01-23 15:44 GMT+00:00 James Pustejovsky <jepusto at gmail.com>:

> Celia,
>
> If you have reasonable prior information about the correlation between
> certain tests, which suggests that some tests are more highly correlated
> then others, then I would definitely recommend using that information.
>
> In principle, you can still use the "impute-the-correlation" strategy even
> while assuming unequal correlations between certain tests. For instance, as
> you suggested, you might assume r = 0.7 for the inter-correlation between
> T1 through T4, but then assume r = 0.2 for the correlation between these
> measures and T5. For sensitivity analysis, you could vary these
> correlations by adding/subtracting 0.1 or 0.2 from each (off-diagonal)
> cell.
>
> Another useful sensitivity analysis would be to assume zero correlations
> between tests from different domains and then also use the struct = "DIAG"
> argument in rma.mv. This amounts to estimating separate (marginal) models
> for each domain. If you get very different average effect estimates than
> you do with the full multivariate model, then it would indicate that the
> results are going to be fairly sensitive to the assumption you make about
> the cross-domain correlations.
>
> Of course, implementing these approaches takes a bit more work. One
> approach would be to create the variance-covariance matrices for each study
> "by hand," and then store them in a list that can be fed into rma.mv.
> This is tedious but might it be the easiest way to go. I have an idea for
> how to make the impute_covariance_matrix() function more helpful for your
> use-case, but it will be a week or two before I can get to it.
>
> James
>
>
>
> On Tue, Jan 23, 2018 at 6:16 AM, Célia Sofia Moreira <
> celiasofiamoreira at gmail.com> wrote:
>
>> Dear all,
>>
>> I am performing a meta-analysis for the first time, and I would be
>> grateful
>> if you could give me some recommendations / suggestions. I apologize for
>> this long description:
>>
>> I have 20 different papers and, because some papers have more than one
>> experiment, 23 experiments are considered, which correspond to 23
>> different
>> samples. The complexity of my case is that each experiment includes
>> different domains, and each domain includes several tests. These domains
>> are latent variables in the individual studies (tests are the indicators).
>> I'm interested in studying the effects in five different domains. Thus,
>> for
>> each domain, I usually collected more than one test type measure.
>>
>> Finally, each sample includes an experimental and a control group. The
>> effects were estimated using Hedges' g (adjusted) and then calculating the
>> difference y between groups. In these calculations, I followed this
>> valuable example:
>> http://www.metafor-project.org/doku.php/analyses:morris2008
>>
>> Unfortunately, papers do not report correlations/covariances between test
>> measures and, so, for each y, I only have the estimated variance v.
>>
>> So, my database looks like:
>>
>> ID ----- Paper --------- Sample ------- Domain ------- Test ------- y ----
>> v  ----  etc.
>>
>> 01 ------- P1  ----------  S1 ----------- D1 ---------- T1 -------- ...
>> 02 ------- P1  ----------  S1 ----------- D1 ---------- T2 -------- ...
>> 03 ------- P1  ----------  S1 ----------- D2 ---------- T3 -------- ...
>> 04 ------- P1  ----------  S1 ----------- D4 ---------- T4 -------- ...
>> 05 ------- P1  ----------  S1 ----------- D4 ---------- T5 -------- ...
>> 06 ------- P1  ----------  S1 ----------- D4 ---------- T6 -------- ...
>>
>> 07 ------- P2  ----------  S2 ----------- D2 ---------- T3 -------- ...
>> 08 ------- P2  ----------  S2 ----------- D2 ---------- T7 -------- ...
>> 09 ------- P2  ----------  S2 ----------- D2 ---------- T8 -------- ...
>> 10 ------- P2  ----------  S2 ----------- D3 ---------- T9 -------- ...
>> 11 ------- P2  ----------  S2 ----------- D4 ---------- T4 -------- ...
>> 12 ------- P2  ----------  S2 ----------- D4 ---------- T5 -------- ...
>> 13 ------- P2  ----------  S2 ----------- D5 ---------- T0 -------- ...
>>
>> 14 ------- P3  ----------  S3 ----------- D1 ---------- T1 -------- ...
>> 15 ------- P3  ----------  S4 ----------- D1 ---------- T1 -------- ...
>> 16 ------- P3  ----------  S4 ----------- D2 ---------- T3 -------- ...
>>
>> .....
>>
>> I would like to assess the significance of these five domains' effects as
>> well as their correlation, using a meta-analyst approach. D1 is the
>> central
>> domain. Thus, I thought about performing a multivariate multilevel random
>> meta-analysis model and then using the correlation matrix between true
>> effects provided in the output. Since I have no sampling covariances, I
>> used the "impute-the-correlation" strategy, by James Pustejovsky, together
>> with robust and sensitive analysis. Because I'm interested in the domains
>> (latent variables), the R code I'm using is the following:
>>
>> Vlist <- impute_covariance_matrix(vi = dat$v, cluster = dat$Sample, r =
>> 0.2)   # Followed by r=.4, r=.6 and r=.8
>> meta1 <- rma.mv(y ~ 0 + Domain, V=Vlist, random = ~ Domain | Sample,
>> struct
>> = "UN", data = dat)
>> meta1r <- robust(meta1, cluster = dat$Sample); summary(meta1r);
>> r1<-cov2cor(meta1r$vb); R1 <- round(c1,2)
>>
>> Does it make sense to you? Do you recommend corrections?
>>
>> Introducing 2 moderators in the model, I got interesting results with this
>> "impute-the-correlation"+robust+sensitive strategy (ranging r from .2 to
>> .8), namely, the significance of D1 true effect.
>>
>> However, the "impute-the-correlation" assumes equal correlations across
>> studies and across outcomes. I've been searching in literature, and I
>> found
>> some papers reporting a high correlation between D2, D3, D4, D5: r=-6-.7
>> between pairs, and r=.7-.8 intra-pairs. Their correlation with D1 is
>> lower,
>> and even lower for one of the pairs.
>>
>> Given this different range of correlations among the outcomes, do you
>> think
>> that the results I got with this strategy are truthful? Or do you think it
>> is better to perform the analysis in different steps, for different
>> subsets
>> of variables in the multivariate multilevel model, for example,
>> considering
>> two at a time, or D1 with each pair separately?
>>
>> Thank you very much for your attention!
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>
>

	[[alternative HTML version deleted]]