[R-meta] Comparing dependent, overlapping correlation coefficients

Wed Aug 22 11:46:27 CEST 2018

Continuing with the earlier toy example:

Let's add a (study-level) moderator 'M' to the dataset:

dat2$dat$M <- c(2,2,2,4,4,4)

res <- rma.mv(yi, dat2$V, mods = ~ var1var2*M, data=dat2$dat)
res

I had to remove the 'random' part because with this little data, the full model isn't going to converge. The output includes:

Model Results:

               estimate      se     zval    pval    ci.lb   ci.ub 
intrcpt          0.0600  0.3134   0.1914  0.8482  -0.5543  0.6743    
var1var2X.Z      0.1100  0.3139   0.3504  0.7260  -0.5052  0.7252    
var1var2Y.Z      0.5400  0.3419   1.5795  0.1142  -0.1301  1.2101    
M                0.0700  0.1022   0.6847  0.4936  -0.1304  0.2704    
var1var2X.Z:M   -0.0050  0.1056  -0.0474  0.9622  -0.2119  0.2019    
var1var2Y.Z:M   -0.1100  0.1111  -0.9900  0.3222  -0.3278  0.1078    

The line with 'var1var2X.Z:M' tells you to what extent the difference between cor(X,Y) and cor(X,Z) is moderated by M (and whether this moderation is significant). 

Best,
Wolfgang

-----Original Message-----
From: Anna-Lena Schubert [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de] 
Sent: Friday, 17 August, 2018 14:51
To: Michael Dewey; Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients

Dear Michael,
but how do I test if the effect of factor levels X vs Y (leaving out Z) is moderated by my continuous covariate? I believe that metafor gives me several ways to test for an overall interaction between a factor and a continuous covariate, but I'm not sure how I'd do follow-up comparisons from there.
Best,
Anna-Lena

Am 17.08.2018 um 14:44 schrieb Michael Dewey:
Dear Anna-Lena 

The concept of interaction works generally for factor by factor, factor by covariate, and covariate by covariate. So you can just go ahead. I must say I have always found it harder to explain the covariate by covariate ones but that may be a defect in my explanatory powers. 

Michael 

On 17/08/2018 12:43, Anna-Lena Schubert wrote: 

Dear Wolfgang, 

thank you so much, this works perfectly well for me! 

I have one final questions before I'm ready to analyze my data: Could I check whether this moderation by variable type is moderated by study characteristics? I.e., is there a way to include an interaction term that again specifically tests if moderator M moderates the difference in correlations between X and Y? I found your example on two categorial moderates and think I could apply that, but most of the moderators I'm thinking about are metric variables. 

Best, 

Anna-Lena 

Am 15.08.2018 um 14:50 schrieb Viechtbauer, Wolfgang (SP): 

library(metafor) 

source("https://gist.githubusercontent.com/wviechtb/700983ab0bde94bed7c645fce770f8e9/raw/5bb5601852b132af533aef41405d58a3ae04cf82/rmat.r") 

dat <- read.table(header=TRUE, text = " 
study    var1 var2  ri  ni 
     1    X    Y    .20  50 
     1    X    Z    .30  50 
     1    Y    Z    .52  50 
     2    X    Y    .34  35 
     2    X    Z    .43  35 
     2    Y    Z    .44  35") 

dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat) 
dat2 

res <- rma.mv(yi, dat2$V, mods = ~ var1var2 - 1, random = ~ var1var2 | id, struct="UN", data=dat2$dat) 
res 

### three contrasts 
anova(res, L=c(1, -1, 0)) 
anova(res, L=c(1, 0, -1)) 
anova(res, L=c(0, 1, -1)) 

You are interested in the first of these three contrasts, that is, whether cor(X,Y) = cor(X,Z). 

Actually, I would recommend to work with: 

dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat, rtoz=TRUE) 
dat2 

This applies Fisher's r-to-z transformation. 

Best, 
Wolfgang 

-----Original Message----- 
From: Anna-Lena Schubert [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de] 
Sent: Wednesday, 15 August, 2018 11:08 
To: Viechtbauer, Wolfgang (SP);r-sig-meta-analysis using r-project.org 
Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients 

Hi Wolfgang, 

thanks so much, I now believe I have sensible values in my V matrix. 
They deviate slightly from manual calculations, but are really close. 

I'm still lost on how to test for the interaction between X and Y then, 
though. I tried 

         res <- rma.mv(yi, V, mods = ~ variable | studyID, data=dat, 
method="ML"), 

but the results don't correspond to the univariate meta-analyses I 
conducted before. In addition, it tests (I believe) for a moderation of 
all three correlations, while I'm only interested in the difference 
between r_XY and r_XZ. Moreover, the manual says "In case the sampling 
errors are correlated, then one can specify the entire 
variance-covariance matrix of the sampling errors via the V argument", 
but it seems I cannot simply leave "yi" out of the argument. 

Best, Anna-Lena 

Am 14.08.2018 um 22:18 schrieb Viechtbauer, Wolfgang (SP): 

You do not need escalc(). The rmat() function gives you the variances along the diagonal of the 'V' matrix. 

The variances should be (1 - ri^2)^2 / (ni - 1). You should be able to double-check that these values correspond to your data. Since ni should be the same for r_XY and r_XZ within a study, then it might be that the variances are roughly the same if the two correlations are not all that different. They should not be identical though (unless r_XY and r_XZ are the same). 
  Best, 
Wolfgang 

-----Original Message----- 
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Anna-Lena Schubert 
Sent: Tuesday, 14 August, 2018 14:20 
To: James Pustejovsky 
Cc:r-sig-meta-analysis using r-project.org 
Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients 

Hi James, 
I used Wolfgang's script on git to calculate the Cov(r_XY, r_XZ) by feeding it Cor(r_YZ). In the next step, I calculated Var(r_XY) and Var(r_XZ) by using the escalc function. However, Var(r_XY) always equals Var(r_XZ) for each study. Does this make sense? 
I nevertheless added all three measures per study into a variance-covariance matrix such as: 
                 r_XY r_XZ    r_XY r_XZ 
r_XY        0.004    0.0001    0    0 
r_XZ        0.0001    0.004    0    0 
r_XY        0    0    0.008    0.002 
r_XZ        0    0    0.002    0.008 
Then, I tried to feed everything into a multivariate meta-analysis: 
     res <- rma.mv(yi, V, mods = ~ variableType - 1, random = ~ variableType | studyNum, struct="UN", data=dat, method="ML") 
The estimates I get for both of the correlation coefficients correspond closely to those I get when only meta-analyzing one of the variable types, which seems great. However, I'm still somewhat concerned that Var(r_XY) = Var(r_XZ). Do you think there may have been some mistake in my code or does it make sense that these variances are equal? 
Best, 
Anna-Lena 
             Am 10.08.2018 um 17:06 schrieb James Pustejovsky: 
Anna-Lena, 

The approach that you suggested (putting the data in "long" format and defining an indicator variable for whether Y or Z is the correlate) is just what I would recommend. However, there is a complication in that the estimates r_XY and r_XZ are correlated (correlated correlation coefficients...say that six times fast!), and the degree of correlation depends on r_YZ. 

1) If you have extracted data on r_YZ then you could use this to compute Cov(r_XY, r_XZ) and then do a multivariate meta-analysis. See discussion here: 
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-January/000483.html 
And this function for computing the required covariance matrices: 
https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9 
There are at least three further alternatives that might be simpler: 

2) If you have r_YZ you could use it to compute the sampling variance of the difference between r_XY and r_XZ, that is: 

Var(r_XY - r_XZ) = Var(r_XY) + Var(r_XZ) - 2 * Cov(r_XY, r_XZ) 

You could then do a univariate meta-analysis on the difference between correlations. 

3) If you do not have r_YZ then you won't be able to estimate Cov(r_XY, r_XZ) very well. You could make a guess about r_YZ and then follow approach (1) or (2) above, using cluster-robust variance estimation to account for the possibly mis-estimated sampling-variance covariance matrix. 

4) Or you could ignore the covariance between r_XY and r_XZ entirely, fit the model to the long data as you describe above, and use cluster-robust variance estimation (clustering by sample) to account for the dependence between r_XY and r_XZ. This is the quickest and dirtiest approach, and the first thing I would try in practice before moving on to the more refined approaches above. 

James 
  On Fri, Aug 10, 2018 at 9:21 AM Anna-Lena Schubert<anna-lena.schubert using psychologie.uni-heidelberg.de>  wrote: 
Dear all, 

I want to run a meta-analysis that compares dependent, overlapping 
correlation coefficients (i.e., I want to see if X correlates more 
strongly with Y than it does with Z). I already ran a meta-analysis 
separately for both of these correlations and would now like to compare 
those two pooled effect sizes statistically. Confidence intervals of the 
two correlations do not overlap (r1 = .18 [.12; .24]; r2 = .32 [.25; 
.39]), but I wonder if there may be a more elegant way to compare these 
correlations than just based on CIs. 

I wonder, for example, if a factorial variable could be used to identify 
those correlations in a "long" data format style, and if I could test 
for a significant interaction between variable type (Y vs. Z) and the 
correlation in a meta-analysis: 

     Study    Variable    r 
     1    Y    .20 
     1    Z    .30 
     2    Y    .34 
     2    Z    .43 

I would greatly appreciate if anyone could tell me if that's a good idea 
or could recommend other approaches. Thanks in advance for any offers of 
help! 

Best, 
Anna-Lena