[R-meta] Calculating covariances in multivariate meta-analysis

Thu Jan 18 00:11:46 CET 2018

Sounds great, thank you. How should I cite rmat()? Is it from the Olkin &
Finn *Psych Bull *that James mentioned earlier? I assume I could probably
cite the metafor package, as well?

On Wed, Jan 17, 2018 at 4:47 PM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> It would take [[2]], but due to the NA not all covariances are computable
> (but the rmat() function should handle this correctly). However, that still
> would not be correct, since the correlations are not all based on the same
> n. So, I would split this up into 2 studies.
>
> -----Original Message-----
> From: Mark White [mailto:markhwhiteii at gmail.com]
> Sent: Wednesday, 17 January, 2018 23:23
> To: Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> Correct. rmat() will *not* take [[2]] however, correct? So I will have to
> split those up into two "studies"? Which should be fine, because I am not
> doubling-up on any participants there.
>
> On Wed, Jan 17, 2018 at 4:21 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> If you use for example [[1]] as input into rmat(), you will get the 6x6
> var-cov matrix of the 6 correlations in that matrix. If I understand you
> correctly, you are interested in cor(auth_muslim, prej_muslim) and
> cor(auth_politician, prej_politician). So, then just take the 2
> rows/columns corresponding to those two correlations.
>
> -----Original Message-----
> From: Mark White [mailto:markhwhiteii at gmail.com]
> Sent: Wednesday, 17 January, 2018 23:10
> To: Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> You will see that 4 - 7 are straightforward, while the first 3 matrices
> are what is giving me trouble.
>
> On Wed, Jan 17, 2018 at 4:08 PM, Mark White <markhwhiteii at gmail.com>
> wrote:
> Wolfgang, I agree that I could split that into two separate studies (those
> n = 20 were just the head of the data set that I chose). But, that does not
> solve the issue of the other two studies where there are dependent effect
> sizes.
>
> James, I am not sure that I follow the logic, although I do agree that a
> multivariate approach may be unnecessarily complicated, if all I'm trying
> to do is estimate the meta-analytic average correlation.
>
> To be a little bit more clear about my data, here is what the correlation
> matrices look like (averaged to the second decimal place, because the 20 x
> 20 matrix was too large to fit otherwise: https://gist.
> github.com/markhwhiteii/5f0eb92096a5ea97f3cce75ef9abafa8. The goal is to
> get the correlations between prejudice and authenticity for the same target
> group (politician, muslim, ksu students, illegal immigrants, etc).
>
> That .txt file is what is returned by the lapply statement here:
> https://github.com/markhwhiteii/dissertation/
> blob/master/R/meta_analysis.R#L110
>
> Best,
> Mark
>
> On Wed, Jan 17, 2018 at 3:56 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> How about treating this as two separate studies (each with n=10), the
> first giving you
>
> cor(ill_imm_auth, ill_imm_prej)
> cor(ill_imm_auth, ksu_prej)
>
> and the second giving you
>
> cor(ksu_auth, ill_imm_prej)
> cor(ksu_auth, ksu_prej)
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Mark White [mailto:markhwhiteii at gmail.com]
> Sent: Wednesday, 17 January, 2018 22:36
> To: Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> Your first question: yes. But one of the other studies has the same
> set-up, but no NAs. And a third has the same set-up, but no NAs, and 10 of
> each type of x and y.
>
> In this case, y are the *_auth variables, and x the *_prej ones.
>
> On Wed, Jan 17, 2018 at 3:33 PM Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> Sry, I can't follow. Are there 20 people, for 10 'ill_imm_auth' has been
> measured, for another 10 'ksu_auth' has been measured? And what corresponds
> to x and y here?
>
> -----Original Message-----
> From: Mark White [mailto:markhwhiteii at gmail.com]
> Sent: Wednesday, 17 January, 2018 22:13
> To: Viechtbauer Wolfgang (SP)
> Cc: James Pustejovsky; r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> Thanks Wolfgang! I'll try to implement that. One of the other issues is
> that, in two of the studies, *everyone* has *every* measure of x_k and y_k
> (where k is the study). However, in another, I observe both types of x_k,
> but only one type of y_k:
>
> # A tibble: 20 x 4
>    ill_imm_auth ksu_auth ill_imm_prej ksu_prej
>           <dbl>    <dbl>        <dbl>    <dbl>
>  1        5.375       NA          4.2      2.0
>  2        1.500       NA          3.4      3.8
>  3        2.875       NA          4.4      3.6
>  4        3.250       NA          4.2      2.6
>  5        4.125       NA          4.6      4.8
>  6        2.750       NA          2.6      3.6
>  7        2.875       NA          2.4      2.4
>  8        6.000       NA          2.8      3.0
>  9        5.875       NA          4.6      2.0
> 10        3.875       NA          3.8      2.4
> 11           NA    1.000          3.4      2.6
> 12           NA    2.375          3.2      4.0
> 13           NA    3.500          3.4      2.4
> 14           NA    1.000          4.0      1.6
> 15           NA    1.000          3.2      1.0
> 16           NA    2.500          2.2      1.6
> 17           NA    1.125          1.0      1.2
> 18           NA    2.500          4.2      1.6
> 19           NA    1.000          1.8      1.0
> 20           NA    2.750          3.0      2.4
>
> Since I just want to know the meta-analytic average (y ~ 1), is there a
> simpler way than making this matrix?
>
> On Wed, Jan 17, 2018 at 3:09 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> Indeed, the computations are a *huge* pain. I wrote this a while ago:
>
> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
>
> It will go into metafor at some point.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-
> bounces at r-project.org] On Behalf Of Mark White
> Sent: Wednesday, 17 January, 2018 19:03
> To: James Pustejovsky
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> James,
>
> Thank you—I have looked at that citation (and similar ones mentioned in the
> documentation for metafor::rma.mv); yes, they do appear to be somewhat of
> a
> pain.
>
> The goal here is simple: All I want is the overall, meta-analytic
> correlation and confidence interval. That is, a multivariate estimate for y
> ~ 1 (i.e., metafor::rma.mv(yi, V)).
>
> Mark
>
> On Wed, Jan 17, 2018 at 11:30 AM, James Pustejovsky <jepusto at gmail.com>
> wrote:
>
> > Mark,
> >
> > The formulas needed to calculate the covariances are given in the
> > following reference:
> >
> > Olkin, I., & Finn, J. (1990). Testing correlated correlations.
> >> Psychological Bulletin, 108(2), 330–333.
> >
> >
> > Unfortunately they're a bit complicated, a pain in the rear to program,
> > and sometimes return non-positive definite covariance matrices that
> create
> > problems at the meta-analysis stage. If you've got the raw data, a
> cleaner
> > approach would be to use a basic bootstrap (i.e., re-sampling cases) for
> > the set of correlations you want to meta-analyze.
> >
> > But a larger question might be relevant here: what is the goal of
> > conducting a multi-variate meta-analysis on these correlations? Is it to
> > come up with a synthetic correlation matrix? To understand heterogeneity
> > across studies in the correlations? Depending on your answer--and given
> > that you have access to the raw data--other statistical approaches (other
> > than MV meta-analysis) might be equally or better suited for the problem.
> >
> > James
> >
> > On Wed, Jan 17, 2018 at 9:17 AM, Mark White <markhwhiteii at gmail.com>
> > wrote:
> >
> >> Hello all,
> >>
> >> I have 8 studies in my dissertation; I want to meta-analyze the
> >> correlation
> >> between focal variable X and outcome Y. Let variables for Study 1 be x1
> >> and
> >> y1, Study 2 be x2 and y2, etc. However, I also have *various
> measurements
> >> *of
> >> each construct in some studies. For example, in Study 1, I have the
> >> correlation between x1_1 and y1_1, as well as x1_2 and y1_2. And in
> Study
> >> 2, I have the correlation between x2_1 and y2_1 as well as x2_2 and
> y2_2.
> >> In Study 3, I have these all the way up to x3_10 and y3_10.
> >>
> >> I want to perform a multivariate meta-analysis, since I have all of the
> >> raw
> >> data. My question: How do I calculate the covariates between these
> >> correlations? I know I want to end up with a covariance matrix where the
> >> diagonal is the variance, off-diagonal the covariances (with all zeros
> >> where they are from different studies). In the analysis examples on the
> >> metafor website, these are already calculated for the user. How do I
> >> calculate these from my raw data?
> >>
> >> Thank you,
> >> Mark
>

	[[alternative HTML version deleted]]