[R-meta] Calculating covariances in multivariate meta-analysis

Wed Jan 17 23:08:55 CET 2018

Wolfgang, I agree that I could split that into two separate studies (those
n = 20 were just the head of the data set that I chose). But, that does not
solve the issue of the other two studies where there are dependent effect
sizes.

James, I am not sure that I follow the logic, although I do agree that a
multivariate approach may be unnecessarily complicated, if all I'm trying
to do is estimate the meta-analytic average correlation.

To be a little bit more clear about my data, here is what the correlation
matrices look like (averaged to the second decimal place, because the 20 x
20 matrix was too large to fit otherwise:
https://gist.github.com/markhwhiteii/5f0eb92096a5ea97f3cce75ef9abafa8. The
goal is to get the correlations between prejudice and authenticity for the
same target group (politician, muslim, ksu students, illegal immigrants,
etc).

That .txt file is what is returned by the lapply statement here:
https://github.com/markhwhiteii/dissertation/blob/master/R/meta_analysis.R#L110

Best,
Mark

On Wed, Jan 17, 2018 at 3:56 PM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

> How about treating this as two separate studies (each with n=10), the
> first giving you
>
> cor(ill_imm_auth, ill_imm_prej)
> cor(ill_imm_auth, ksu_prej)
>
> and the second giving you
>
> cor(ksu_auth, ill_imm_prej)
> cor(ksu_auth, ksu_prej)
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Mark White [mailto:markhwhiteii at gmail.com]
> Sent: Wednesday, 17 January, 2018 22:36
> To: Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> Your first question: yes. But one of the other studies has the same
> set-up, but no NAs. And a third has the same set-up, but no NAs, and 10 of
> each type of x and y.
>
> In this case, y are the *_auth variables, and x the *_prej ones.
>
> On Wed, Jan 17, 2018 at 3:33 PM Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> Sry, I can't follow. Are there 20 people, for 10 'ill_imm_auth' has been
> measured, for another 10 'ksu_auth' has been measured? And what corresponds
> to x and y here?
>
> -----Original Message-----
> From: Mark White [mailto:markhwhiteii at gmail.com]
> Sent: Wednesday, 17 January, 2018 22:13
> To: Viechtbauer Wolfgang (SP)
> Cc: James Pustejovsky; r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> Thanks Wolfgang! I'll try to implement that. One of the other issues is
> that, in two of the studies, *everyone* has *every* measure of x_k and y_k
> (where k is the study). However, in another, I observe both types of x_k,
> but only one type of y_k:
>
> # A tibble: 20 x 4
>    ill_imm_auth ksu_auth ill_imm_prej ksu_prej
>           <dbl>    <dbl>        <dbl>    <dbl>
>  1        5.375       NA          4.2      2.0
>  2        1.500       NA          3.4      3.8
>  3        2.875       NA          4.4      3.6
>  4        3.250       NA          4.2      2.6
>  5        4.125       NA          4.6      4.8
>  6        2.750       NA          2.6      3.6
>  7        2.875       NA          2.4      2.4
>  8        6.000       NA          2.8      3.0
>  9        5.875       NA          4.6      2.0
> 10        3.875       NA          3.8      2.4
> 11           NA    1.000          3.4      2.6
> 12           NA    2.375          3.2      4.0
> 13           NA    3.500          3.4      2.4
> 14           NA    1.000          4.0      1.6
> 15           NA    1.000          3.2      1.0
> 16           NA    2.500          2.2      1.6
> 17           NA    1.125          1.0      1.2
> 18           NA    2.500          4.2      1.6
> 19           NA    1.000          1.8      1.0
> 20           NA    2.750          3.0      2.4
>
> Since I just want to know the meta-analytic average (y ~ 1), is there a
> simpler way than making this matrix?
>
> On Wed, Jan 17, 2018 at 3:09 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
> Indeed, the computations are a *huge* pain. I wrote this a while ago:
>
> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
>
> It will go into metafor at some point.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-
> bounces at r-project.org] On Behalf Of Mark White
> Sent: Wednesday, 17 January, 2018 19:03
> To: James Pustejovsky
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] Calculating covariances in multivariate meta-analysis
>
> James,
>
> Thank you—I have looked at that citation (and similar ones mentioned in the
> documentation for metafor::rma.mv); yes, they do appear to be somewhat of
> a
> pain.
>
> The goal here is simple: All I want is the overall, meta-analytic
> correlation and confidence interval. That is, a multivariate estimate for y
> ~ 1 (i.e., metafor::rma.mv(yi, V)).
>
> Mark
>
> On Wed, Jan 17, 2018 at 11:30 AM, James Pustejovsky <jepusto at gmail.com>
> wrote:
>
> > Mark,
> >
> > The formulas needed to calculate the covariances are given in the
> > following reference:
> >
> > Olkin, I., & Finn, J. (1990). Testing correlated correlations.
> >> Psychological Bulletin, 108(2), 330–333.
> >
> >
> > Unfortunately they're a bit complicated, a pain in the rear to program,
> > and sometimes return non-positive definite covariance matrices that
> create
> > problems at the meta-analysis stage. If you've got the raw data, a
> cleaner
> > approach would be to use a basic bootstrap (i.e., re-sampling cases) for
> > the set of correlations you want to meta-analyze.
> >
> > But a larger question might be relevant here: what is the goal of
> > conducting a multi-variate meta-analysis on these correlations? Is it to
> > come up with a synthetic correlation matrix? To understand heterogeneity
> > across studies in the correlations? Depending on your answer--and given
> > that you have access to the raw data--other statistical approaches (other
> > than MV meta-analysis) might be equally or better suited for the problem.
> >
> > James
> >
> > On Wed, Jan 17, 2018 at 9:17 AM, Mark White <markhwhiteii at gmail.com>
> > wrote:
> >
> >> Hello all,
> >>
> >> I have 8 studies in my dissertation; I want to meta-analyze the
> >> correlation
> >> between focal variable X and outcome Y. Let variables for Study 1 be x1
> >> and
> >> y1, Study 2 be x2 and y2, etc. However, I also have *various
> measurements
> >> *of
> >> each construct in some studies. For example, in Study 1, I have the
> >> correlation between x1_1 and y1_1, as well as x1_2 and y1_2. And in
> Study
> >> 2, I have the correlation between x2_1 and y2_1 as well as x2_2 and
> y2_2.
> >> In Study 3, I have these all the way up to x3_10 and y3_10.
> >>
> >> I want to perform a multivariate meta-analysis, since I have all of the
> >> raw
> >> data. My question: How do I calculate the covariates between these
> >> correlations? I know I want to end up with a covariance matrix where the
> >> diagonal is the variance, off-diagonal the covariances (with all zeros
> >> where they are from different studies). In the analysis examples on the
> >> metafor website, these are already calculated for the user. How do I
> >> calculate these from my raw data?
> >>
> >> Thank you,
> >> Mark
>

	[[alternative HTML version deleted]]