[R-meta] Comparing dependent, overlapping correlation coefficients
Anna-Lena Schubert
@nn@-len@@@chubert @ending from p@ychologie@uni-heidelberg@de
Mon Oct 1 14:47:09 CEST 2018
I have recently tried to finish all analyses and have realized that I
still haven't fully understood how moderation works here. In the example
below, I'm interested in whether the difference between cor(X,Y) and
cor(X,Z) is moderated by some factor. From my reading of the metafor
examples, var1var2X.Z:M should indicate whether the cor(X,Z) is
moderated by M. However, Wolfgang says that term instead describes a
moderation of the difference between correlations. I'm perfectly happy
to believe him, but I would really like to understand why this is the case.
In my empirical example the results look as follows:
estimate se zval pval ci.lb ci.ub
var1var2X.Y 0.6580 0.0218 30.1818 <.0001 0.6152 0.7007 ***
var1var2X.Z -0.1282 0.0468 -2.7420 0.0061 -0.2199 -0.0366 **
var1var2Y.Z -0.2896 0.0480 -6.0315 <.0001 -0.3837 -0.1955 ***
M1 0.0736 0.0301 2.4474 0.0144 0.0147 0.1326 *
var1var2X.Z:M1 -0.1648 0.0786 -2.0969 0.0360 -0.3188 -0.0108 *
var1var2Y.Z:M1 -0.1266 0.0845 -1.4983 0.1340 -0.2922 0.0390
Here I'm interested in the difference between cor(x,z) and cor(y,z). Which of these interaction terms is critical to the moderation of this correlation, and why? Also, how can I interpret the direction of the effect?
If I look at another display of the same results, I get an understanding of the moderation, but I'm not sure about a clear test here. Would anova(res(0,1,-1,0,1,-1)) be a sensible test? What I'd really want to test would be if X.Z:M0 - Y.Z:M0 = X.Z:M1 - Y.Z:M1
estimate se zval pval ci.lb ci.ub
var1var2X.Y:M0 0.6580 0.0218 30.1818 <.0001 0.6152 0.7007 ***
var1var2X.Z:M0 -0.1282 0.0468 -2.7420 0.0061 -0.2199 -0.0366 **
var1var2Y.Z:M0 -0.2896 0.0480 -6.0315 <.0001 -0.3837 -0.1955 ***
var1var2X.Y:M1 0.7316 0.0207 35.3007 <.0001 0.6910 0.7722 ***
var1var2X.Z:M1 -0.2194 0.0483 -4.5437 <.0001 -0.3140 -0.1248 ***
var1var2Y.Z:M1 -0.3426 0.0491 -6.9767 <.0001 -0.4388 -0.2463 ***
Am 22.08.2018 um 11:46 schrieb Viechtbauer, Wolfgang (SP):
> Continuing with the earlier toy example:
>
> Let's add a (study-level) moderator 'M' to the dataset:
>
> dat2$dat$M <- c(2,2,2,4,4,4)
>
> res <- rma.mv(yi, dat2$V, mods = ~ var1var2*M, data=dat2$dat)
> res
>
> I had to remove the 'random' part because with this little data, the full model isn't going to converge. The output includes:
>
> Model Results:
>
> estimate se zval pval ci.lb ci.ub
> intrcpt 0.0600 0.3134 0.1914 0.8482 -0.5543 0.6743
> var1var2X.Z 0.1100 0.3139 0.3504 0.7260 -0.5052 0.7252
> var1var2Y.Z 0.5400 0.3419 1.5795 0.1142 -0.1301 1.2101
> M 0.0700 0.1022 0.6847 0.4936 -0.1304 0.2704
> var1var2X.Z:M -0.0050 0.1056 -0.0474 0.9622 -0.2119 0.2019
> var1var2Y.Z:M -0.1100 0.1111 -0.9900 0.3222 -0.3278 0.1078
>
> The line with 'var1var2X.Z:M' tells you to what extent the difference between cor(X,Y) and cor(X,Z) is moderated by M (and whether this moderation is significant).
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Anna-Lena Schubert [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de]
> Sent: Friday, 17 August, 2018 14:51
> To: Michael Dewey; Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>
> Dear Michael,
> but how do I test if the effect of factor levels X vs Y (leaving out Z) is moderated by my continuous covariate? I believe that metafor gives me several ways to test for an overall interaction between a factor and a continuous covariate, but I'm not sure how I'd do follow-up comparisons from there.
> Best,
> Anna-Lena
>
> Am 17.08.2018 um 14:44 schrieb Michael Dewey:
> Dear Anna-Lena
>
> The concept of interaction works generally for factor by factor, factor by covariate, and covariate by covariate. So you can just go ahead. I must say I have always found it harder to explain the covariate by covariate ones but that may be a defect in my explanatory powers.
>
> Michael
>
> On 17/08/2018 12:43, Anna-Lena Schubert wrote:
>
> Dear Wolfgang,
>
> thank you so much, this works perfectly well for me!
>
> I have one final questions before I'm ready to analyze my data: Could I check whether this moderation by variable type is moderated by study characteristics? I.e., is there a way to include an interaction term that again specifically tests if moderator M moderates the difference in correlations between X and Y? I found your example on two categorial moderates and think I could apply that, but most of the moderators I'm thinking about are metric variables.
>
> Best,
>
> Anna-Lena
>
>
> Am 15.08.2018 um 14:50 schrieb Viechtbauer, Wolfgang (SP):
>
> library(metafor)
>
> source("https://gist.githubusercontent.com/wviechtb/700983ab0bde94bed7c645fce770f8e9/raw/5bb5601852b132af533aef41405d58a3ae04cf82/rmat.r")
>
> dat <- read.table(header=TRUE, text = "
> study var1 var2 ri ni
> 1 X Y .20 50
> 1 X Z .30 50
> 1 Y Z .52 50
> 2 X Y .34 35
> 2 X Z .43 35
> 2 Y Z .44 35")
>
> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat)
> dat2
>
> res <- rma.mv(yi, dat2$V, mods = ~ var1var2 - 1, random = ~ var1var2 | id, struct="UN", data=dat2$dat)
> res
>
> ### three contrasts
> anova(res, L=c(1, -1, 0))
> anova(res, L=c(1, 0, -1))
> anova(res, L=c(0, 1, -1))
>
> You are interested in the first of these three contrasts, that is, whether cor(X,Y) = cor(X,Z).
>
> Actually, I would recommend to work with:
>
> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat, rtoz=TRUE)
> dat2
>
> This applies Fisher's r-to-z transformation.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Anna-Lena Schubert [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de]
> Sent: Wednesday, 15 August, 2018 11:08
> To: Viechtbauer, Wolfgang (SP);r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>
> Hi Wolfgang,
>
> thanks so much, I now believe I have sensible values in my V matrix.
> They deviate slightly from manual calculations, but are really close.
>
> I'm still lost on how to test for the interaction between X and Y then,
> though. I tried
>
> res <- rma.mv(yi, V, mods = ~ variable | studyID, data=dat,
> method="ML"),
>
> but the results don't correspond to the univariate meta-analyses I
> conducted before. In addition, it tests (I believe) for a moderation of
> all three correlations, while I'm only interested in the difference
> between r_XY and r_XZ. Moreover, the manual says "In case the sampling
> errors are correlated, then one can specify the entire
> variance-covariance matrix of the sampling errors via the V argument",
> but it seems I cannot simply leave "yi" out of the argument.
>
> Best, Anna-Lena
>
> Am 14.08.2018 um 22:18 schrieb Viechtbauer, Wolfgang (SP):
>
> You do not need escalc(). The rmat() function gives you the variances along the diagonal of the 'V' matrix.
>
> The variances should be (1 - ri^2)^2 / (ni - 1). You should be able to double-check that these values correspond to your data. Since ni should be the same for r_XY and r_XZ within a study, then it might be that the variances are roughly the same if the two correlations are not all that different. They should not be identical though (unless r_XY and r_XZ are the same).
> Best,
> Wolfgang
>
> -----Original Message-----
> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Anna-Lena Schubert
> Sent: Tuesday, 14 August, 2018 14:20
> To: James Pustejovsky
> Cc:r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>
> Hi James,
> I used Wolfgang's script on git to calculate the Cov(r_XY, r_XZ) by feeding it Cor(r_YZ). In the next step, I calculated Var(r_XY) and Var(r_XZ) by using the escalc function. However, Var(r_XY) always equals Var(r_XZ) for each study. Does this make sense?
> I nevertheless added all three measures per study into a variance-covariance matrix such as:
> r_XY r_XZ r_XY r_XZ
> r_XY 0.004 0.0001 0 0
> r_XZ 0.0001 0.004 0 0
> r_XY 0 0 0.008 0.002
> r_XZ 0 0 0.002 0.008
> Then, I tried to feed everything into a multivariate meta-analysis:
> res <- rma.mv(yi, V, mods = ~ variableType - 1, random = ~ variableType | studyNum, struct="UN", data=dat, method="ML")
> The estimates I get for both of the correlation coefficients correspond closely to those I get when only meta-analyzing one of the variable types, which seems great. However, I'm still somewhat concerned that Var(r_XY) = Var(r_XZ). Do you think there may have been some mistake in my code or does it make sense that these variances are equal?
> Best,
> Anna-Lena
> Am 10.08.2018 um 17:06 schrieb James Pustejovsky:
> Anna-Lena,
>
> The approach that you suggested (putting the data in "long" format and defining an indicator variable for whether Y or Z is the correlate) is just what I would recommend. However, there is a complication in that the estimates r_XY and r_XZ are correlated (correlated correlation coefficients...say that six times fast!), and the degree of correlation depends on r_YZ.
>
> 1) If you have extracted data on r_YZ then you could use this to compute Cov(r_XY, r_XZ) and then do a multivariate meta-analysis. See discussion here:
> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-January/000483.html
> And this function for computing the required covariance matrices:
> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
> There are at least three further alternatives that might be simpler:
>
> 2) If you have r_YZ you could use it to compute the sampling variance of the difference between r_XY and r_XZ, that is:
>
> Var(r_XY - r_XZ) = Var(r_XY) + Var(r_XZ) - 2 * Cov(r_XY, r_XZ)
>
> You could then do a univariate meta-analysis on the difference between correlations.
>
> 3) If you do not have r_YZ then you won't be able to estimate Cov(r_XY, r_XZ) very well. You could make a guess about r_YZ and then follow approach (1) or (2) above, using cluster-robust variance estimation to account for the possibly mis-estimated sampling-variance covariance matrix.
>
> 4) Or you could ignore the covariance between r_XY and r_XZ entirely, fit the model to the long data as you describe above, and use cluster-robust variance estimation (clustering by sample) to account for the dependence between r_XY and r_XZ. This is the quickest and dirtiest approach, and the first thing I would try in practice before moving on to the more refined approaches above.
>
> James
> On Fri, Aug 10, 2018 at 9:21 AM Anna-Lena Schubert<anna-lena.schubert using psychologie.uni-heidelberg.de> wrote:
> Dear all,
>
> I want to run a meta-analysis that compares dependent, overlapping
> correlation coefficients (i.e., I want to see if X correlates more
> strongly with Y than it does with Z). I already ran a meta-analysis
> separately for both of these correlations and would now like to compare
> those two pooled effect sizes statistically. Confidence intervals of the
> two correlations do not overlap (r1 = .18 [.12; .24]; r2 = .32 [.25;
> .39]), but I wonder if there may be a more elegant way to compare these
> correlations than just based on CIs.
>
> I wonder, for example, if a factorial variable could be used to identify
> those correlations in a "long" data format style, and if I could test
> for a significant interaction between variable type (Y vs. Z) and the
> correlation in a meta-analysis:
>
> Study Variable r
> 1 Y .20
> 1 Z .30
> 2 Y .34
> 2 Z .43
>
> I would greatly appreciate if anyone could tell me if that's a good idea
> or could recommend other approaches. Thanks in advance for any offers of
> help!
>
> Best,
> Anna-Lena
--
Signatur
Dr. Anna-Lena Schubert
Postdoc at Section of Personality
Heidelberg University - Institute of Psychology
Hauptstraße 47-51
D-69117 Heidelberg Germany
Phone: +49 6221 54 7746
Mail: anna-lena.schubert using psychologie.uni-heidelberg.de
Web: http://www.psychologie.uni-heidelberg.de/ae/diff/diff/people-schubert.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20181001/8c8b95ee/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgkkopgkhooekjdg.png
Type: image/png
Size: 25980 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20181001/8c8b95ee/attachment-0001.png>
More information about the R-sig-meta-analysis
mailing list