[R-meta] Comparing dependent, overlapping correlation coefficients
Anna-Lena Schubert
@nn@-len@@@chubert @ending from p@ychologie@uni-heidelberg@de
Fri Aug 17 13:43:39 CEST 2018
Dear Wolfgang,
thank you so much, this works perfectly well for me!
I have one final questions before I'm ready to analyze my data: Could I
check whether this moderation by variable type is moderated by study
characteristics? I.e., is there a way to include an interaction term
that again specifically tests if moderator M moderates the difference in
correlations between X and Y? I found your example on two categorial
moderates and think I could apply that, but most of the moderators I'm
thinking about are metric variables.
Best,
Anna-Lena
Am 15.08.2018 um 14:50 schrieb Viechtbauer, Wolfgang (SP):
> library(metafor)
>
> source("https://gist.githubusercontent.com/wviechtb/700983ab0bde94bed7c645fce770f8e9/raw/5bb5601852b132af533aef41405d58a3ae04cf82/rmat.r")
>
> dat <- read.table(header=TRUE, text = "
> study var1 var2 ri ni
> 1 X Y .20 50
> 1 X Z .30 50
> 1 Y Z .52 50
> 2 X Y .34 35
> 2 X Z .43 35
> 2 Y Z .44 35")
>
> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat)
> dat2
>
> res <- rma.mv(yi, dat2$V, mods = ~ var1var2 - 1, random = ~ var1var2 | id, struct="UN", data=dat2$dat)
> res
>
> ### three contrasts
> anova(res, L=c(1, -1, 0))
> anova(res, L=c(1, 0, -1))
> anova(res, L=c(0, 1, -1))
>
> You are interested in the first of these three contrasts, that is, whether cor(X,Y) = cor(X,Z).
>
> Actually, I would recommend to work with:
>
> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat, rtoz=TRUE)
> dat2
>
> This applies Fisher's r-to-z transformation.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Anna-Lena Schubert [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de]
> Sent: Wednesday, 15 August, 2018 11:08
> To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>
> Hi Wolfgang,
>
> thanks so much, I now believe I have sensible values in my V matrix.
> They deviate slightly from manual calculations, but are really close.
>
> I'm still lost on how to test for the interaction between X and Y then,
> though. I tried
>
> res <- rma.mv(yi, V, mods = ~ variable | studyID, data=dat,
> method="ML"),
>
> but the results don't correspond to the univariate meta-analyses I
> conducted before. In addition, it tests (I believe) for a moderation of
> all three correlations, while I'm only interested in the difference
> between r_XY and r_XZ. Moreover, the manual says "In case the sampling
> errors are correlated, then one can specify the entire
> variance-covariance matrix of the sampling errors via the V argument",
> but it seems I cannot simply leave "yi" out of the argument.
>
> Best, Anna-Lena
>
> Am 14.08.2018 um 22:18 schrieb Viechtbauer, Wolfgang (SP):
>> You do not need escalc(). The rmat() function gives you the variances along the diagonal of the 'V' matrix.
>>
>> The variances should be (1 - ri^2)^2 / (ni - 1). You should be able to double-check that these values correspond to your data. Since ni should be the same for r_XY and r_XZ within a study, then it might be that the variances are roughly the same if the two correlations are not all that different. They should not be identical though (unless r_XY and r_XZ are the same).
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Anna-Lena Schubert
>> Sent: Tuesday, 14 August, 2018 14:20
>> To: James Pustejovsky
>> Cc: r-sig-meta-analysis using r-project.org
>> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>>
>> Hi James,
>> I used Wolfgang's script on git to calculate the Cov(r_XY, r_XZ) by feeding it Cor(r_YZ). In the next step, I calculated Var(r_XY) and Var(r_XZ) by using the escalc function. However, Var(r_XY) always equals Var(r_XZ) for each study. Does this make sense?
>> I nevertheless added all three measures per study into a variance-covariance matrix such as:
>> r_XY r_XZ r_XY r_XZ
>> r_XY 0.004 0.0001 0 0
>> r_XZ 0.0001 0.004 0 0
>> r_XY 0 0 0.008 0.002
>> r_XZ 0 0 0.002 0.008
>> Then, I tried to feed everything into a multivariate meta-analysis:
>> res <- rma.mv(yi, V, mods = ~ variableType - 1, random = ~ variableType | studyNum, struct="UN", data=dat, method="ML")
>> The estimates I get for both of the correlation coefficients correspond closely to those I get when only meta-analyzing one of the variable types, which seems great. However, I'm still somewhat concerned that Var(r_XY) = Var(r_XZ). Do you think there may have been some mistake in my code or does it make sense that these variances are equal?
>> Best,
>> Anna-Lena
>>
>> Am 10.08.2018 um 17:06 schrieb James Pustejovsky:
>> Anna-Lena,
>>
>> The approach that you suggested (putting the data in "long" format and defining an indicator variable for whether Y or Z is the correlate) is just what I would recommend. However, there is a complication in that the estimates r_XY and r_XZ are correlated (correlated correlation coefficients...say that six times fast!), and the degree of correlation depends on r_YZ.
>>
>> 1) If you have extracted data on r_YZ then you could use this to compute Cov(r_XY, r_XZ) and then do a multivariate meta-analysis. See discussion here:
>> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-January/000483.html
>> And this function for computing the required covariance matrices:
>> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
>> There are at least three further alternatives that might be simpler:
>>
>> 2) If you have r_YZ you could use it to compute the sampling variance of the difference between r_XY and r_XZ, that is:
>>
>> Var(r_XY - r_XZ) = Var(r_XY) + Var(r_XZ) - 2 * Cov(r_XY, r_XZ)
>>
>> You could then do a univariate meta-analysis on the difference between correlations.
>>
>> 3) If you do not have r_YZ then you won't be able to estimate Cov(r_XY, r_XZ) very well. You could make a guess about r_YZ and then follow approach (1) or (2) above, using cluster-robust variance estimation to account for the possibly mis-estimated sampling-variance covariance matrix.
>>
>> 4) Or you could ignore the covariance between r_XY and r_XZ entirely, fit the model to the long data as you describe above, and use cluster-robust variance estimation (clustering by sample) to account for the dependence between r_XY and r_XZ. This is the quickest and dirtiest approach, and the first thing I would try in practice before moving on to the more refined approaches above.
>>
>> James
>>
>> On Fri, Aug 10, 2018 at 9:21 AM Anna-Lena Schubert <anna-lena.schubert using psychologie.uni-heidelberg.de> wrote:
>> Dear all,
>>
>> I want to run a meta-analysis that compares dependent, overlapping
>> correlation coefficients (i.e., I want to see if X correlates more
>> strongly with Y than it does with Z). I already ran a meta-analysis
>> separately for both of these correlations and would now like to compare
>> those two pooled effect sizes statistically. Confidence intervals of the
>> two correlations do not overlap (r1 = .18 [.12; .24]; r2 = .32 [.25;
>> .39]), but I wonder if there may be a more elegant way to compare these
>> correlations than just based on CIs.
>>
>> I wonder, for example, if a factorial variable could be used to identify
>> those correlations in a "long" data format style, and if I could test
>> for a significant interaction between variable type (Y vs. Z) and the
>> correlation in a meta-analysis:
>>
>> Study Variable r
>> 1 Y .20
>> 1 Z .30
>> 2 Y .34
>> 2 Z .43
>>
>> I would greatly appreciate if anyone could tell me if that's a good idea
>> or could recommend other approaches. Thanks in advance for any offers of
>> help!
>>
>> Best,
>> Anna-Lena
--
Signatur
Dr. Anna-Lena Schubert
Postdoc at Section of Personality
Heidelberg University - Institute of Psychology
Hauptstraße 47-51
D-69117 Heidelberg Germany
Phone: +49 6221 54 7746
Mail: anna-lena.schubert using psychologie.uni-heidelberg.de
Web: http://www.psychologie.uni-heidelberg.de/ae/diff/diff/people-schubert.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180817/28f204b1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kdhllfoeliodffnb.png
Type: image/png
Size: 25980 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180817/28f204b1/attachment-0001.png>
More information about the R-sig-meta-analysis
mailing list