# [R-meta] Calculating covariances in multivariate meta-analysis

Viechtbauer Wolfgang (SP) wolfgang.viechtbauer at maastrichtuniversity.nl
Wed Jan 17 23:47:45 CET 2018

```It would take [[2]], but due to the NA not all covariances are computable (but the rmat() function should handle this correctly). However, that still would not be correct, since the correlations are not all based on the same n. So, I would split this up into 2 studies.

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com]
Sent: Wednesday, 17 January, 2018 23:23
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

Correct. rmat() will *not* take [[2]] however, correct? So I will have to split those up into two "studies"? Which should be fine, because I am not doubling-up on any participants there.

On Wed, Jan 17, 2018 at 4:21 PM, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
If you use for example [[1]] as input into rmat(), you will get the 6x6 var-cov matrix of the 6 correlations in that matrix. If I understand you correctly, you are interested in cor(auth_muslim, prej_muslim) and cor(auth_politician, prej_politician). So, then just take the 2 rows/columns corresponding to those two correlations.

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com]
Sent: Wednesday, 17 January, 2018 23:10
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

You will see that 4 - 7 are straightforward, while the first 3 matrices are what is giving me trouble.

On Wed, Jan 17, 2018 at 4:08 PM, Mark White <markhwhiteii at gmail.com> wrote:
Wolfgang, I agree that I could split that into two separate studies (those n = 20 were just the head of the data set that I chose). But, that does not solve the issue of the other two studies where there are dependent effect sizes.

James, I am not sure that I follow the logic, although I do agree that a multivariate approach may be unnecessarily complicated, if all I'm trying to do is estimate the meta-analytic average correlation.

To be a little bit more clear about my data, here is what the correlation matrices look like (averaged to the second decimal place, because the 20 x 20 matrix was too large to fit otherwise: https://gist.github.com/markhwhiteii/5f0eb92096a5ea97f3cce75ef9abafa8. The goal is to get the correlations between prejudice and authenticity for the same target group (politician, muslim, ksu students, illegal immigrants, etc).

That .txt file is what is returned by the lapply statement here: https://github.com/markhwhiteii/dissertation/blob/master/R/meta_analysis.R#L110

Best,
Mark

On Wed, Jan 17, 2018 at 3:56 PM, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
How about treating this as two separate studies (each with n=10), the first giving you

cor(ill_imm_auth, ill_imm_prej)
cor(ill_imm_auth, ksu_prej)

and the second giving you

cor(ksu_auth, ill_imm_prej)
cor(ksu_auth, ksu_prej)

Best,
Wolfgang

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com]
Sent: Wednesday, 17 January, 2018 22:36
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

Your first question: yes. But one of the other studies has the same set-up, but no NAs. And a third has the same set-up, but no NAs, and 10 of each type of x and y.

In this case, y are the *_auth variables, and x the *_prej ones.

On Wed, Jan 17, 2018 at 3:33 PM Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
Sry, I can't follow. Are there 20 people, for 10 'ill_imm_auth' has been measured, for another 10 'ksu_auth' has been measured? And what corresponds to x and y here?

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com]
Sent: Wednesday, 17 January, 2018 22:13
To: Viechtbauer Wolfgang (SP)
Cc: James Pustejovsky; r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

Thanks Wolfgang! I'll try to implement that. One of the other issues is that, in two of the studies, *everyone* has *every* measure of x_k and y_k (where k is the study). However, in another, I observe both types of x_k, but only one type of y_k:

# A tibble: 20 x 4
ill_imm_auth ksu_auth ill_imm_prej ksu_prej
<dbl>    <dbl>        <dbl>    <dbl>
1        5.375       NA          4.2      2.0
2        1.500       NA          3.4      3.8
3        2.875       NA          4.4      3.6
4        3.250       NA          4.2      2.6
5        4.125       NA          4.6      4.8
6        2.750       NA          2.6      3.6
7        2.875       NA          2.4      2.4
8        6.000       NA          2.8      3.0
9        5.875       NA          4.6      2.0
10        3.875       NA          3.8      2.4
11           NA    1.000          3.4      2.6
12           NA    2.375          3.2      4.0
13           NA    3.500          3.4      2.4
14           NA    1.000          4.0      1.6
15           NA    1.000          3.2      1.0
16           NA    2.500          2.2      1.6
17           NA    1.125          1.0      1.2
18           NA    2.500          4.2      1.6
19           NA    1.000          1.8      1.0
20           NA    2.750          3.0      2.4

Since I just want to know the meta-analytic average (y ~ 1), is there a simpler way than making this matrix?

On Wed, Jan 17, 2018 at 3:09 PM, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
Indeed, the computations are a *huge* pain. I wrote this a while ago:

https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9

It will go into metafor at some point.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Mark White
Sent: Wednesday, 17 January, 2018 19:03
To: James Pustejovsky
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

James,

Thank you—I have looked at that citation (and similar ones mentioned in the
documentation for metafor::rma.mv); yes, they do appear to be somewhat of a
pain.

The goal here is simple: All I want is the overall, meta-analytic
correlation and confidence interval. That is, a multivariate estimate for y
~ 1 (i.e., metafor::rma.mv(yi, V)).

Mark

On Wed, Jan 17, 2018 at 11:30 AM, James Pustejovsky <jepusto at gmail.com>
wrote:

> Mark,
>
> The formulas needed to calculate the covariances are given in the
> following reference:
>
> Olkin, I., & Finn, J. (1990). Testing correlated correlations.
>> Psychological Bulletin, 108(2), 330–333.
>
>
> Unfortunately they're a bit complicated, a pain in the rear to program,
> and sometimes return non-positive definite covariance matrices that create
> problems at the meta-analysis stage. If you've got the raw data, a cleaner
> approach would be to use a basic bootstrap (i.e., re-sampling cases) for
> the set of correlations you want to meta-analyze.
>
> But a larger question might be relevant here: what is the goal of
> conducting a multi-variate meta-analysis on these correlations? Is it to
> come up with a synthetic correlation matrix? To understand heterogeneity
> that you have access to the raw data--other statistical approaches (other
> than MV meta-analysis) might be equally or better suited for the problem.
>
> James
>
> On Wed, Jan 17, 2018 at 9:17 AM, Mark White <markhwhiteii at gmail.com>
> wrote:
>
>> Hello all,
>>
>> I have 8 studies in my dissertation; I want to meta-analyze the
>> correlation
>> between focal variable X and outcome Y. Let variables for Study 1 be x1
>> and
>> y1, Study 2 be x2 and y2, etc. However, I also have *various measurements
>> *of
>> each construct in some studies. For example, in Study 1, I have the
>> correlation between x1_1 and y1_1, as well as x1_2 and y1_2. And in Study
>> 2, I have the correlation between x2_1 and y2_1 as well as x2_2 and y2_2.
>> In Study 3, I have these all the way up to x3_10 and y3_10.
>>
>> I want to perform a multivariate meta-analysis, since I have all of the
>> raw
>> data. My question: How do I calculate the covariates between these
>> correlations? I know I want to end up with a covariance matrix where the
>> diagonal is the variance, off-diagonal the covariances (with all zeros
>> where they are from different studies). In the analysis examples on the
>> metafor website, these are already calculated for the user. How do I
>> calculate these from my raw data?
>>
>> Thank you,
>> Mark
```