[R-meta] Calculating covariances in multivariate meta-analysis

Wed Jan 17 23:10:14 CET 2018

You will see that 4 - 7 are straightforward, while the first 3 matrices are
what is giving me trouble.

On Wed, Jan 17, 2018 at 4:08 PM, Mark White <markhwhiteii at gmail.com> wrote:

> Wolfgang, I agree that I could split that into two separate studies (those
> n = 20 were just the head of the data set that I chose). But, that does not
> solve the issue of the other two studies where there are dependent effect
> sizes.
>
> James, I am not sure that I follow the logic, although I do agree that a
> multivariate approach may be unnecessarily complicated, if all I'm trying
> to do is estimate the meta-analytic average correlation.
>
> To be a little bit more clear about my data, here is what the correlation
> matrices look like (averaged to the second decimal place, because the 20 x
> 20 matrix was too large to fit otherwise: https://gist.
> github.com/markhwhiteii/5f0eb92096a5ea97f3cce75ef9abafa8. The goal is to
> get the correlations between prejudice and authenticity for the same target
> group (politician, muslim, ksu students, illegal immigrants, etc).
>
> That .txt file is what is returned by the lapply statement here:
> https://github.com/markhwhiteii/dissertation/
> blob/master/R/meta_analysis.R#L110
>
> Best,
> Mark
>
> On Wed, Jan 17, 2018 at 3:56 PM, Viechtbauer Wolfgang (SP) <
> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>
>> How about treating this as two separate studies (each with n=10), the
>> first giving you
>>
>> cor(ill_imm_auth, ill_imm_prej)
>> cor(ill_imm_auth, ksu_prej)
>>
>> and the second giving you
>>
>> cor(ksu_auth, ill_imm_prej)
>> cor(ksu_auth, ksu_prej)
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: Mark White [mailto:markhwhiteii at gmail.com]
>> Sent: Wednesday, 17 January, 2018 22:36
>> To: Viechtbauer Wolfgang (SP)
>> Cc: r-sig-meta-analysis at r-project.org
>> Subject: Re: [R-meta] Calculating covariances in multivariate
>> meta-analysis
>>
>> Your first question: yes. But one of the other studies has the same
>> set-up, but no NAs. And a third has the same set-up, but no NAs, and 10 of
>> each type of x and y.
>>
>> In this case, y are the *_auth variables, and x the *_prej ones.
>>
>> On Wed, Jan 17, 2018 at 3:33 PM Viechtbauer Wolfgang (SP) <
>> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>> Sry, I can't follow. Are there 20 people, for 10 'ill_imm_auth' has been
>> measured, for another 10 'ksu_auth' has been measured? And what corresponds
>> to x and y here?
>>
>> -----Original Message-----
>> From: Mark White [mailto:markhwhiteii at gmail.com]
>> Sent: Wednesday, 17 January, 2018 22:13
>> To: Viechtbauer Wolfgang (SP)
>> Cc: James Pustejovsky; r-sig-meta-analysis at r-project.org
>> Subject: Re: [R-meta] Calculating covariances in multivariate
>> meta-analysis
>>
>> Thanks Wolfgang! I'll try to implement that. One of the other issues is
>> that, in two of the studies, *everyone* has *every* measure of x_k and y_k
>> (where k is the study). However, in another, I observe both types of x_k,
>> but only one type of y_k:
>>
>> # A tibble: 20 x 4
>>    ill_imm_auth ksu_auth ill_imm_prej ksu_prej
>>           <dbl>    <dbl>        <dbl>    <dbl>
>>  1        5.375       NA          4.2      2.0
>>  2        1.500       NA          3.4      3.8
>>  3        2.875       NA          4.4      3.6
>>  4        3.250       NA          4.2      2.6
>>  5        4.125       NA          4.6      4.8
>>  6        2.750       NA          2.6      3.6
>>  7        2.875       NA          2.4      2.4
>>  8        6.000       NA          2.8      3.0
>>  9        5.875       NA          4.6      2.0
>> 10        3.875       NA          3.8      2.4
>> 11           NA    1.000          3.4      2.6
>> 12           NA    2.375          3.2      4.0
>> 13           NA    3.500          3.4      2.4
>> 14           NA    1.000          4.0      1.6
>> 15           NA    1.000          3.2      1.0
>> 16           NA    2.500          2.2      1.6
>> 17           NA    1.125          1.0      1.2
>> 18           NA    2.500          4.2      1.6
>> 19           NA    1.000          1.8      1.0
>> 20           NA    2.750          3.0      2.4
>>
>> Since I just want to know the meta-analytic average (y ~ 1), is there a
>> simpler way than making this matrix?
>>
>> On Wed, Jan 17, 2018 at 3:09 PM, Viechtbauer Wolfgang (SP) <
>> wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
>> Indeed, the computations are a *huge* pain. I wrote this a while ago:
>>
>> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
>>
>> It will go into metafor at some point.
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bo
>> unces at r-project.org] On Behalf Of Mark White
>> Sent: Wednesday, 17 January, 2018 19:03
>> To: James Pustejovsky
>> Cc: r-sig-meta-analysis at r-project.org
>> Subject: Re: [R-meta] Calculating covariances in multivariate
>> meta-analysis
>>
>> James,
>>
>> Thank you—I have looked at that citation (and similar ones mentioned in
>> the
>> documentation for metafor::rma.mv); yes, they do appear to be somewhat
>> of a
>> pain.
>>
>> The goal here is simple: All I want is the overall, meta-analytic
>> correlation and confidence interval. That is, a multivariate estimate for
>> y
>> ~ 1 (i.e., metafor::rma.mv(yi, V)).
>>
>> Mark
>>
>> On Wed, Jan 17, 2018 at 11:30 AM, James Pustejovsky <jepusto at gmail.com>
>> wrote:
>>
>> > Mark,
>> >
>> > The formulas needed to calculate the covariances are given in the
>> > following reference:
>> >
>> > Olkin, I., & Finn, J. (1990). Testing correlated correlations.
>> >> Psychological Bulletin, 108(2), 330–333.
>> >
>> >
>> > Unfortunately they're a bit complicated, a pain in the rear to program,
>> > and sometimes return non-positive definite covariance matrices that
>> create
>> > problems at the meta-analysis stage. If you've got the raw data, a
>> cleaner
>> > approach would be to use a basic bootstrap (i.e., re-sampling cases) for
>> > the set of correlations you want to meta-analyze.
>> >
>> > But a larger question might be relevant here: what is the goal of
>> > conducting a multi-variate meta-analysis on these correlations? Is it to
>> > come up with a synthetic correlation matrix? To understand heterogeneity
>> > across studies in the correlations? Depending on your answer--and given
>> > that you have access to the raw data--other statistical approaches
>> (other
>> > than MV meta-analysis) might be equally or better suited for the
>> problem.
>> >
>> > James
>> >
>> > On Wed, Jan 17, 2018 at 9:17 AM, Mark White <markhwhiteii at gmail.com>
>> > wrote:
>> >
>> >> Hello all,
>> >>
>> >> I have 8 studies in my dissertation; I want to meta-analyze the
>> >> correlation
>> >> between focal variable X and outcome Y. Let variables for Study 1 be x1
>> >> and
>> >> y1, Study 2 be x2 and y2, etc. However, I also have *various
>> measurements
>> >> *of
>> >> each construct in some studies. For example, in Study 1, I have the
>> >> correlation between x1_1 and y1_1, as well as x1_2 and y1_2. And in
>> Study
>> >> 2, I have the correlation between x2_1 and y2_1 as well as x2_2 and
>> y2_2.
>> >> In Study 3, I have these all the way up to x3_10 and y3_10.
>> >>
>> >> I want to perform a multivariate meta-analysis, since I have all of the
>> >> raw
>> >> data. My question: How do I calculate the covariates between these
>> >> correlations? I know I want to end up with a covariance matrix where
>> the
>> >> diagonal is the variance, off-diagonal the covariances (with all zeros
>> >> where they are from different studies). In the analysis examples on the
>> >> metafor website, these are already calculated for the user. How do I
>> >> calculate these from my raw data?
>> >>
>> >> Thank you,
>> >> Mark
>>
>
>

	[[alternative HTML version deleted]]