[R-meta] Calculating covariances in multivariate meta-analysis

[Viechtbauer Wolfgang (SP)]

Please keep the list cc-ed.

Can't say whether it looks right -- I would have to go through every line step-by-step with the actual data.

Using rmat() with rtoz=TRUE should give different values (just tested this to make sure). But if the correlations are not too far away from 0, then the difference between the raw and r-to-z transformed values could be so small that it ends up not making much of a difference (but you should still see minor differences).

Whether it is ok to meta-analyze raw correlation coefficients (or as some have put it: whether it is wrong to meta-analyze r-to-z transformed coefficients) has been debated for a long time in the literature. I personally would go with r-to-z transformed values, although if you then do the back-transformation, you will get an estimate of the median correlation, not the mean correlation. If you want to get the mean, you have to do a different transformation, as described in:

Hafdahl, A. R. (2009). Improved Fisher z estimators for univariate random-effects meta-analysis of correlations. British Journal of Mathematical and Statistical Psychology, 62(2), 233-261.

Hafdahl, A. R. (2010). Random-effects meta-analysis of correlations: Monte Carlo evaluation of mean estimators. British Journal of Mathematical and Statistical Psychology, 63(1), 227-254.

Hafdahl, A. R., & Williams, M. A. (2009). Meta-analysis of correlations revisited: Attempted replication and extension of Field's (2001) simulation studies. Psychological Methods, 14(1), 24-42.

I call this the 'integral' transformation, and there is transf.ztor.int() in metafor to do this. It's directly applicable to models fitted with rma(). Not sure about multilevel/multivariate models though.

Best,
Wolfgang

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com] 
Sent: Thursday, 18 January, 2018 1:06
To: Viechtbauer Wolfgang (SP)
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

Finished it here: https://github.com/markhwhiteii/dissertation/blob/master/R/meta_analysis.R#L114-L148

Does that look about right to you? I only selected the 18 correlations I am interested in. The thing that concerns me is that making rtoz = TRUE doesn't change the results. Is it OK to submit raw correlations to rmat and then to rma.mv? Or is it changing to z regardless of if I hit TRUE or FALSE on that?

Thanks again,
Mark

On Wed, Jan 17, 2018 at 5:11 PM, Mark White <markhwhiteii at gmail.com> wrote:
Sounds great, thank you. How should I cite rmat()? Is it from the Olkin & Finn Psych Bull that James mentioned earlier? I assume I could probably cite the metafor package, as well?

On Wed, Jan 17, 2018 at 4:47 PM, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
It would take [[2]], but due to the NA not all covariances are computable (but the rmat() function should handle this correctly). However, that still would not be correct, since the correlations are not all based on the same n. So, I would split this up into 2 studies.

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com]
Sent: Wednesday, 17 January, 2018 23:23
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

Correct. rmat() will *not* take [[2]] however, correct? So I will have to split those up into two "studies"? Which should be fine, because I am not doubling-up on any participants there.

On Wed, Jan 17, 2018 at 4:21 PM, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
If you use for example [[1]] as input into rmat(), you will get the 6x6 var-cov matrix of the 6 correlations in that matrix. If I understand you correctly, you are interested in cor(auth_muslim, prej_muslim) and cor(auth_politician, prej_politician). So, then just take the 2 rows/columns corresponding to those two correlations.

-----Original Message-----
From: Mark White [mailto:markhwhiteii at gmail.com]
Sent: Wednesday, 17 January, 2018 23:10
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis

You will see that 4 - 7 are straightforward, while the first 3 matrices are what is giving me trouble.

On Wed, Jan 17, 2018 at 4:08 PM, Mark White <markhwhiteii at gmail.com> wrote:
Wolfgang, I agree that I could split that into two separate studies (those n = 20 were just the head of the data set that I chose). But, that does not solve the issue of the other two studies where there are dependent effect sizes.

James, I am not sure that I follow the logic, although I do agree that a multivariate approach may be unnecessarily complicated, if all I'm trying to do is estimate the meta-analytic average correlation.

To be a little bit more clear about my data, here is what the correlation matrices look like (averaged to the second decimal place, because the 20 x 20 matrix was too large to fit otherwise: https://gist.github.com/markhwhiteii/5f0eb92096a5ea97f3cce75ef9abafa8. The goal is to get the correlations between prejudice and authenticity for the same target group (politician, muslim, ksu students, illegal immigrants, etc).

That .txt file is what is returned by the lapply statement here: https://github.com/markhwhiteii/dissertation/blob/master/R/meta_analysis.R#L110

Best,
Mark