[R-meta] Computing var-covariance matrix with correlations of six non-independent outcomes
James Pustejovsky
jepu@to @end|ng |rom gm@||@com
Tue Jul 7 22:37:08 CEST 2020
Hi Mika
Thanks for sharing this example data. One housekeeping note: you'll need to
code missing values in the correlation matrices as NA, not "NA". Using "NA"
causes the numeric values to be converted to character strings. After
you've done so, the next thing will be to remove the parts of the
correlation matrices that are missing, so that the matrix for study 1
corresponds to just the effect sizes that you have for study 1. You can do
that with the following code:
# filter out NA values
remove_NAs <- function(x) {
NA_rows <- apply(x, 1, function(r) all(is.na(r)))
x[!NA_rows, !NA_rows, drop = FALSE]
}
corlist_complete <- lapply(corlist, remove_NAs)
Okay then on to the meat of your question. The usual formula for the
covariance between SMDs of different outcomes with a common sample (as
given in e.g., Gleser & Olkin, 2009, Handbook of Research Synthesis and
Meta-Analysis) involves the degrees of freedom of the standard deviation
used in the denominator of the SMD. For example, if the SD is pooled across
group A and group B, then the degrees of freedom would be df = nA + nB - 2,
where nA and nB are the sample sizes of the two groups: Say that d_j and
d_k are the SMD estimates for outcomes j and k, respectively. Let r_jk be
the correlation between outcomes j and k. Then the covariance between the
effect size estimates is
Cov(d_j, d_k) = r_jk (1 / nA + 1 / nB) + (r_jk)^2 * d_j * d_k / (2 * df)
You haven't provided the sample size information in your data, so it's not
possible to use this formula. However, the covariance can be quite closely
approximated by assuming that the correlation between the effect size
estimates is the same as the correlation between the outcomes. Let V_j and
V_k be the sampling variances of the SMD estimates for outcomes j and k,
respectively. Then a pretty close approximation is
Cov(d_j, d_k) ~= r_jk * sqrt(V_j * V_k)
This can be calculated using the data that you've provided. Here's example
code:
# make a list with a vector of variances for each study
Varlist <- with(meta_dat, split(v, study))
calculate_cov_matrix <- function(v, r_mat) {
tcrossprod(sqrt(v)) * r_mat
}
Vcov_matrices <- mapply(calculate_cov_matrix, v = Varlist, r_mat =
corlist_complete)
# check that we can recover the original correlation matrices
lapply(Vcov_matrices, cov2cor)
all.equal(lapply(Vcov_matrices, cov2cor), corlist_complete,
check.attributes = FALSE)
You can then use Vcov_matrices in the V argument of rma.mv() to fit a
multvariate meta-analysis model.
James
On Tue, Jul 7, 2020 at 2:53 PM Mika Manninen <mixu89 using gmail.com> wrote:
> James,
>
> Thank you so much for replying.
>
> The dataset (8 studies out of 25 included) can be created with the
> following code:
>
> ###Creating vectors for the outcome, study, motivation, effect size, and
> variance estimate
>
> #Total effects/outcomes
> outcome <- c(1:28)
>
> #Study
> study <- c(1,1,1,1,1,2,2,2,2,3,3,3,3,3,4,4,4,4,4,5,6,7,8,8,8,8,8,8)
>
> #Six different motivation types measured in the studies
> motivation <- c(1,3,4,5,6,1,3,4,5,1,3,4,5,6,1,3,4,5,6,6,6,6,1,2,3,4,5,6)
>
>
> #Hedges g:s
> g <- c(0.6068, 0.0603, 0.2684, -0.0886, 0.0415, 1.592, 1.4031, 0.7928,
> 0.2013, 0.541, 0.1169,
> 0.3129, -0.0275, -0.3536, 1.5886, 2.7218, -1.6273, -0.4375,
> -1.0695, -0.1247, -0.1038,
> -0.2706, 0.2045, -0.2701, 0.3763, -0.7371, -0.0666, 0.2789)
>
> #Variance estimates
> v <- c( 0.0162, 0.0155, 0.0157, 0.0155, 0.0155, 0.1889,
> 0.17875984,0.15484225,
> 0.14432401, 0.0329, 0.0318, 0.0322, 0.0318, 0.0323, 0.1886, 0.2758,
> 0.1909, 0.147, 0.164, 0.0067, 0.0028, 0.004, 0.0726, 0.0729,
> 0.0735,
> 0.0771, 0.0723, 0.073)
>
> ###Dataset matrix with different levels and motivation types
> meta <- cbind(outcome, study, motivation, g, v)
> View(meta)
>
> I created 6x6 correlation matrices (with 15 unique correlations) for the
> eight included studies. I am not sure if this is the most sensible approach
> as only a few studies have measured all six outcomes. Value NA reflects the
> absence of a correlation (absence of that one or both of the motivation
> types in the study). The row and column numbers (1-6) correspond to the
> variable motivation in the created dataset.
>
> ### Correlation matrices for studies 1-8
>
> study1c <- rbind(c(1, "NA", .96, .34, -.33, -.66), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c(.96, "NA", 1, .55, -.12, -.50),
> c(.34, "NA", .55, 1, .53, .05), c(-.33, "NA", -.12, .53,
> 1, .72), c(-.66, "NA", -.50, .05, .72, 1))
>
> study2c <- rbind(c(1, "NA", .96, .34, -.33, "NA"), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c(.96, "NA", 1, .55, -.12, "NA"),
> c(.34, "NA", .55, 1, .53, "NA"), c(-.33, "NA", -.12, .53,
> 1, "NA"), c("NA", "NA", "NA", "NA", "NA", "NA"))
>
> study3c <- rbind(c(1, "NA", .96, .34, -.33, -.66), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c(.96, "NA", 1, .55, -.12, -.50),
> c(.34, "NA", .55, 1, .53, .05), c(-.33, "NA", -.12, .53,
> 1, .72), c(-.66, "NA", -.50, .05, .72, 1))
>
> study4c <- rbind(c(1, "NA", .85, .35, .06, -.44), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c(.85, "NA", 1, .59, .46, -.14),
> c(.35, "NA", .59, 1, .71, .27), c(.06, "NA", .46, .71, 1,
> .52), c(-.44, "NA", -.14, .27, .52, 1))
>
> study5c <- rbind(c("NA", "NA", "NA", "NA", "NA", "NA"), c("NA", "NA",
> "NA", "NA", "NA", "NA"), c("NA", "NA", "NA", "NA", "NA", "NA"),
> c("NA", "NA", "NA", "NA", "NA", "NA"), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c("NA", "NA", "NA", "NA", "NA", 1))
>
> study6c <- rbind(c("NA", "NA", "NA", "NA", "NA", "NA"), c("NA", "NA",
> "NA", "NA", "NA", "NA"), c("NA", "NA", "NA", "NA", "NA", "NA"),
> c("NA", "NA", "NA", "NA", "NA", "NA"), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c("NA", "NA", "NA", "NA", "NA", 1))
>
> study7c <- rbind(c("NA", "NA", "NA", "NA", "NA", "NA"), c("NA", "NA",
> "NA", "NA", "NA", "NA"), c("NA", "NA", "NA", "NA", "NA", "NA"),
> c("NA", "NA", "NA", "NA", "NA", "NA"), c("NA", "NA", "NA",
> "NA", "NA", "NA"), c("NA", "NA", "NA", "NA", "NA", 1))
>
> study8c <- rbind(c(1, .79, .84, .24, -.20, -.50), c(.79, 1, .93, .47, .00,
> -.31), c(.84, .93, 1, .57, -.07, -.52),
> c(.24, .47, .57, 1, .43, .01), c(-.20, .00, -.07, .43,
> 1, .55), c(-.50, -.31, -.52, .01, .55, 1))
>
> #list with all the correlations matrices
>
> corlist <- list(study1c, study2c, study3c, study4c, study5c, study6c,
> study7c, study8c)
>
> Mika
>
> ti 7. heinäk. 2020 klo 19.54 James Pustejovsky (jepusto using gmail.com)
> kirjoitti:
>
>> Hi Mika,
>>
>> To add to Wolfgang's question, could you tell us a little bit more about
>> how you have structured the data on correlations between types of
>> motivation? Is it just one correlation matrix (6X6 matrix, with 1 + 2 + 3 +
>> 4 + 5 = 15 unique correlations)? Or is it study-specific?
>>
>> This sort of calculation is a bit tricky to carry out so I am not
>> surprised that you haven't found a solution in the listserv archives. If
>> you are willing to share your dataset (or a subset thereof, say 3-4 studies
>> worth of data), it may make it easier for us to help problem solve.
>>
>> James
>>
>> On Tue, Jul 7, 2020 at 11:42 AM Viechtbauer, Wolfgang (SP) <
>> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>>> Dear Mika,
>>>
>>> What effect size measure are you using for the meta-analysis?
>>>
>>> Best,
>>> Wolfgang
>>>
>>> >-----Original Message-----
>>> >From: R-sig-meta-analysis [mailto:
>>> r-sig-meta-analysis-bounces using r-project.org]
>>> >On Behalf Of Mika Manninen
>>> >Sent: Tuesday, 07 July, 2020 18:10
>>> >To: r-sig-meta-analysis using r-project.org
>>> >Subject: [R-meta] Computing var-covariance matrix with correlations of
>>> six
>>> >non-independent outcomes
>>> >
>>> >Hello,
>>> >
>>> >I am doing a meta-analysis looking at the effect of a teaching
>>> intervention
>>> >(versus control) on six types of motivation/behavioral regulation.
>>> >Theoretically and empirically these constructs form a continuum in which
>>> >the continuum neighbors are most strongly positively correlated and the
>>> >furthest from one another most negatively correlated.
>>> >
>>> >I have 95 effects. These effects come from 25 studies, each reporting
>>> >scores for between 1-6 motivation types. The number of effects per
>>> >motivation ranges from 22 to 13. In some studies, they have measured
>>> only
>>> >one or two types whereas in others they have measured 5 or all 6 types
>>> of
>>> >motivation.
>>> >
>>> >I originally ran a separate random-effects meta-analysis for all the six
>>> >outcomes. However, I got feedback that the dependency of the motivation
>>> >types should be taken into account and a 3-level meta-analysis was
>>> >recommended. After looking into it, the 3-level model seems to be a
>>> worse
>>> >approach than the multivariate approach.
>>> >
>>> >As is not usually the case, I have succeeded in gathering all
>>> correlations
>>> >between all the motivation types for all studies (some from original
>>> >reporting and some from previous meta-analysis findings).
>>> >
>>> >My question is, how do I compute the V-matrix for this data in order to
>>> run
>>> >the multivariate analysis? I read the whole archive but I could not
>>> find a
>>> >clear answer to the problem.
>>> >
>>> >Thank you very much in advance,
>>> >
>>> >Mika
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
>>
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list