[R-meta] A further question about 'correlations' in multivariate model: correlation in observed effects vs. correlation in true effects

[James Pustejovsky]

Hi Yefeng, I'll add a couple of observations to Wolfgang's response.

First let me address your question about whether there is a concept of
correlation in observed effect size estimates, and whether this is
meaningful. I think the term is ambiguous--it most likely means the
sampling correlation between estimates, conditional on the true effects for
a given study. This is as Wolfgang noted. But it might also refer to an
_overall_ correlation between the effect size estimates in a study. This
correlation is what you would observe if you just created a scatterplot of
the effect size estimates for the first outcome versus the effect size
estimates for the second outcome.

Say that for each study:
T_1j is the observed effect size estimate for the cognition outcome in
study j
V_1j is the sampling variance of T_1j
T_2j is the observed effect size estimate for the anxiety outcome in study j
V_2j is the sampling variance of T_2j
r is the sampling correlation (the "within-study" correlation) between T_1j
and T_2j
Then if you plotted T_1j versus T_2j, the correlation between the points is
an _overall_ correlation, which _combines_ the within-study correlation in
the sampling errors and the "between-study" correlation in true effect
sizes.

If you are willing to work under a particular set of modeling assumptions,
such as the bivariate meta-analysis model that you've described, then you
can derive this overall correlation between observed effect size estimates
from the same study. Say that the true effect sizes for study j are
theta_1j and theta_2j, and we assume an unstructured covariance matrix
where Var(theta_1j) = tau_1, Var(theta_2j) = tau_2, and Cor(theta_1j,
theta_2j) = rho. Then
Cov(T_1j, T_2j) = r * sqrt(V_1j V_2j) + rho * sqrt(tau_1 * tau_2)
Var(T_1j) = V_1j + tau_1
Var(T_2j) = V_2j + tau_2
and so
Cor(T_1j, T_2j) = [r * sqrt(V_1j V_2j) + rho * sqrt(tau_1 * tau_2)] /
sqrt[(V_1j + tau_1) * (V_2j + tau_2)]
As you can see, this is a rather complicated expression that depends on
several parameters and also varies from study to study. This suggests it's
hard to interpret in substantive terms.

If we make some further simplifying assumptions that the sampling variances
are equal, V_1j = V_2j = V_j, and the between-study variances are equal,
tau_1 = tau_2 = tau, then things simplify a bit:
Cor(T_1j, T_2j) = (r * V_j + rho * tau) / (V_j + tau) = r * (1 - phi_j) +
rho * phi_j,
where phi_j = tau / (V_j + tau), a sort of study-specific version of
I-squared. So with this expression, we can see that the overall correlation
is an average of the correlation between sampling errors and the
correlation between the random effects.

Richard Riley and colleagues have some work from a while back looking at
how one could estimate the bivariate meta-analytic model by assuming that
the correlation between sampling errors and the correlation between random
effects is the same (i.e., r = rho):
Riley, R. D., Thompson, J. R., & Abrams, K. R. (2008). An alternative model
for bivariate random-effects meta-analysis when the within-study
correlations are unknown. *Biostatistics*, *9*(1), 172-186.
Generally though, there's not any reason why this would need to be the case.

James
https://jepusto.com

On Thu, May 5, 2022 at 4:51 AM Viechtbauer, Wolfgang (NP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Yefeng,
>
> Please see below for my responses.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Yefeng Yang
> >Sent: Thursday, 05 May, 2022 9:57
> >To: r-sig-meta-analysis using r-project.org
> >Subject: [R-meta] A further question about 'correlations' in multivariate
> model:
> >correlation in observed effects vs. correlation in true effects
> >
> >Hi subscribers,
> >
> >Excited to be here again. Last time, I asked one question regarding the
> within-
> >study correlation (used for the variance-covariance matrix of sampling
> errors of
> >effect size estimates) and the correlation in *true* effects (estimated
> from the
> >variance-covariance matrix of random effect). Please refer to my question
> here:
> >https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2022-May/004026.html.
> >
> >Regarding my question, Wolfgang gave me a fairly clear answer, which
> resolved my
> >long-standing confusion. I appreciated this very much. See his answer
> here:
> >https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2022-May/004029.html.
> >
> >Now, I have further, albeit small, questions about the relationship
> between
> >correlation in observed effects and that in true effects (these questions
> were
> >inspired by "variance in the observed effect and that in the true
> effect"):
> >
> >  1.   is there a concept of correlation in observed effects? I mean
> whether
> >correlation in observed effects is meaningful?  If yes, what is its
> general
> >meaning or implication? You can use an example of one study containing two
> >outcomes (cognition and anxiety) to explain it to me.
>
> This is the within-study correlation I talked about.
>
> >  2.  what is the relationship between correlation in observed effects
> and that
> >in true effects? I am expecting a quantitative relationship between the
> two (just
> >like the variance of the observed effect =  variance of true effect +
> sampling
> >variance)
>
> There is no inherent relationship between the two. It could be that the
> within-study correlations are positive but that the correlation in the
> underlying true effects are negative or vice-versa, or that the true
> effects are not correlated at all, and so on.
>
> >  3.  any parameters in the meta-analytic model (see below for an
> example) can
> >reflect or denote the correlation in the observed effect (if the
> corresponding
> >concept exists; see question 1): rma.mv(yi, V, mods = ~ outcome, random
> = ~
> >outcome | study, struct="UN", data=mydata)
>
> I don't understand the question.
>
> >  4.  correlation in variance-covariance matrix of sampling variance can
> be
> >formally called within-study correlation. This is fairly clear. I wonder
> what is
> >the formal name or general name of correlation (rho) in the
> variance-covariance
> >of a random effect (for example rho in the structure like "UN")
>
> Some might call it the between-study correlation, but to me this phrasing
> is a bit odd, because it sounds a bit like the correlation denotes how
> something in study A correlates with something in study B, but that is not
> what it denotes (it denotes how the true effects correlate within a study,
> so in that sense, it is also a within-study correlation, but let's not call
> it that to avoid confusion).
>
> >  5.  in contrast to the within-study correlation, there is also another
> >correlation, named the between-study correlation involved in the
> multivariate
> >meta-analytic model (see Riley R D, Abrams K R, Sutton A J, et al.
> Bivariate
> >random-effects meta-analysis and the estimation of between-study
> correlation[J].
> >BMC Medical Research Methodology, 2007, 7(1): 1-15.). So what is the
> meaning of
> >between-study correlation? Is between-study correlation exactly the
> correlation
> >in true effects (as you explained in the meta-analytic context, we have
> multiple
> >studies, not only one study for the within-study correlation)?
>
> Yes - see 4.
>
> >I would be grateful if would like to take the time to clarify my
> confusion.
> >
> >Best,
> >
> >Yefeng Yang PhD
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]