[R-meta] Comparing dependent, overlapping correlation coefficients
Michael Dewey
li@t@ @ending from dewey@myzen@co@uk
Fri Aug 17 14:44:17 CEST 2018
Dear Anna-Lena
The concept of interaction works generally for factor by factor, factor
by covariate, and covariate by covariate. So you can just go ahead. I
must say I have always found it harder to explain the covariate by
covariate ones but that may be a defect in my explanatory powers.
Michael
On 17/08/2018 12:43, Anna-Lena Schubert wrote:
> Dear Wolfgang,
>
> thank you so much, this works perfectly well for me!
>
> I have one final questions before I'm ready to analyze my data: Could I
> check whether this moderation by variable type is moderated by study
> characteristics? I.e., is there a way to include an interaction term
> that again specifically tests if moderator M moderates the difference in
> correlations between X and Y? I found your example on two categorial
> moderates and think I could apply that, but most of the moderators I'm
> thinking about are metric variables.
>
> Best,
>
> Anna-Lena
>
>
> Am 15.08.2018 um 14:50 schrieb Viechtbauer, Wolfgang (SP):
>> library(metafor)
>>
>> source("https://gist.githubusercontent.com/wviechtb/700983ab0bde94bed7c645fce770f8e9/raw/5bb5601852b132af533aef41405d58a3ae04cf82/rmat.r")
>>
>> dat <- read.table(header=TRUE, text = "
>> study var1 var2 ri ni
>> 1 X Y .20 50
>> 1 X Z .30 50
>> 1 Y Z .52 50
>> 2 X Y .34 35
>> 2 X Z .43 35
>> 2 Y Z .44 35")
>>
>> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat)
>> dat2
>>
>> res <- rma.mv(yi, dat2$V, mods = ~ var1var2 - 1, random = ~ var1var2 | id, struct="UN", data=dat2$dat)
>> res
>>
>> ### three contrasts
>> anova(res, L=c(1, -1, 0))
>> anova(res, L=c(1, 0, -1))
>> anova(res, L=c(0, 1, -1))
>>
>> You are interested in the first of these three contrasts, that is, whether cor(X,Y) = cor(X,Z).
>>
>> Actually, I would recommend to work with:
>>
>> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat, rtoz=TRUE)
>> dat2
>>
>> This applies Fisher's r-to-z transformation.
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: Anna-Lena Schubert [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de]
>> Sent: Wednesday, 15 August, 2018 11:08
>> To: Viechtbauer, Wolfgang (SP);r-sig-meta-analysis using r-project.org
>> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>>
>> Hi Wolfgang,
>>
>> thanks so much, I now believe I have sensible values in my V matrix.
>> They deviate slightly from manual calculations, but are really close.
>>
>> I'm still lost on how to test for the interaction between X and Y then,
>> though. I tried
>>
>> res <- rma.mv(yi, V, mods = ~ variable | studyID, data=dat,
>> method="ML"),
>>
>> but the results don't correspond to the univariate meta-analyses I
>> conducted before. In addition, it tests (I believe) for a moderation of
>> all three correlations, while I'm only interested in the difference
>> between r_XY and r_XZ. Moreover, the manual says "In case the sampling
>> errors are correlated, then one can specify the entire
>> variance-covariance matrix of the sampling errors via the V argument",
>> but it seems I cannot simply leave "yi" out of the argument.
>>
>> Best, Anna-Lena
>>
>> Am 14.08.2018 um 22:18 schrieb Viechtbauer, Wolfgang (SP):
>>> You do not need escalc(). The rmat() function gives you the variances along the diagonal of the 'V' matrix.
>>>
>>> The variances should be (1 - ri^2)^2 / (ni - 1). You should be able to double-check that these values correspond to your data. Since ni should be the same for r_XY and r_XZ within a study, then it might be that the variances are roughly the same if the two correlations are not all that different. They should not be identical though (unless r_XY and r_XZ are the same).
>>>
>>> Best,
>>> Wolfgang
>>>
>>> -----Original Message-----
>>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Anna-Lena Schubert
>>> Sent: Tuesday, 14 August, 2018 14:20
>>> To: James Pustejovsky
>>> Cc:r-sig-meta-analysis using r-project.org
>>> Subject: Re: [R-meta] Comparing dependent, overlapping correlation coefficients
>>>
>>> Hi James,
>>> I used Wolfgang's script on git to calculate the Cov(r_XY, r_XZ) by feeding it Cor(r_YZ). In the next step, I calculated Var(r_XY) and Var(r_XZ) by using the escalc function. However, Var(r_XY) always equals Var(r_XZ) for each study. Does this make sense?
>>> I nevertheless added all three measures per study into a variance-covariance matrix such as:
>>> r_XY r_XZ r_XY r_XZ
>>> r_XY 0.004 0.0001 0 0
>>> r_XZ 0.0001 0.004 0 0
>>> r_XY 0 0 0.008 0.002
>>> r_XZ 0 0 0.002 0.008
>>> Then, I tried to feed everything into a multivariate meta-analysis:
>>> res <- rma.mv(yi, V, mods = ~ variableType - 1, random = ~ variableType | studyNum, struct="UN", data=dat, method="ML")
>>> The estimates I get for both of the correlation coefficients correspond closely to those I get when only meta-analyzing one of the variable types, which seems great. However, I'm still somewhat concerned that Var(r_XY) = Var(r_XZ). Do you think there may have been some mistake in my code or does it make sense that these variances are equal?
>>> Best,
>>> Anna-Lena
>>>
>>> Am 10.08.2018 um 17:06 schrieb James Pustejovsky:
>>> Anna-Lena,
>>>
>>> The approach that you suggested (putting the data in "long" format and defining an indicator variable for whether Y or Z is the correlate) is just what I would recommend. However, there is a complication in that the estimates r_XY and r_XZ are correlated (correlated correlation coefficients...say that six times fast!), and the degree of correlation depends on r_YZ.
>>>
>>> 1) If you have extracted data on r_YZ then you could use this to compute Cov(r_XY, r_XZ) and then do a multivariate meta-analysis. See discussion here:
>>> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-January/000483.html
>>> And this function for computing the required covariance matrices:
>>> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
>>> There are at least three further alternatives that might be simpler:
>>>
>>> 2) If you have r_YZ you could use it to compute the sampling variance of the difference between r_XY and r_XZ, that is:
>>>
>>> Var(r_XY - r_XZ) = Var(r_XY) + Var(r_XZ) - 2 * Cov(r_XY, r_XZ)
>>>
>>> You could then do a univariate meta-analysis on the difference between correlations.
>>>
>>> 3) If you do not have r_YZ then you won't be able to estimate Cov(r_XY, r_XZ) very well. You could make a guess about r_YZ and then follow approach (1) or (2) above, using cluster-robust variance estimation to account for the possibly mis-estimated sampling-variance covariance matrix.
>>>
>>> 4) Or you could ignore the covariance between r_XY and r_XZ entirely, fit the model to the long data as you describe above, and use cluster-robust variance estimation (clustering by sample) to account for the dependence between r_XY and r_XZ. This is the quickest and dirtiest approach, and the first thing I would try in practice before moving on to the more refined approaches above.
>>>
>>> James
>>>
>>> On Fri, Aug 10, 2018 at 9:21 AM Anna-Lena Schubert<anna-lena.schubert using psychologie.uni-heidelberg.de> wrote:
>>> Dear all,
>>>
>>> I want to run a meta-analysis that compares dependent, overlapping
>>> correlation coefficients (i.e., I want to see if X correlates more
>>> strongly with Y than it does with Z). I already ran a meta-analysis
>>> separately for both of these correlations and would now like to compare
>>> those two pooled effect sizes statistically. Confidence intervals of the
>>> two correlations do not overlap (r1 = .18 [.12; .24]; r2 = .32 [.25;
>>> .39]), but I wonder if there may be a more elegant way to compare these
>>> correlations than just based on CIs.
>>>
>>> I wonder, for example, if a factorial variable could be used to identify
>>> those correlations in a "long" data format style, and if I could test
>>> for a significant interaction between variable type (Y vs. Z) and the
>>> correlation in a meta-analysis:
>>>
>>> Study Variable r
>>> 1 Y .20
>>> 1 Z .30
>>> 2 Y .34
>>> 2 Z .43
>>>
>>> I would greatly appreciate if anyone could tell me if that's a good idea
>>> or could recommend other approaches. Thanks in advance for any offers of
>>> help!
>>>
>>> Best,
>>> Anna-Lena
>
> --
> Signatur
>
>
> Dr. Anna-Lena Schubert
>
> Postdoc at Section of Personality
> Heidelberg University - Institute of Psychology
>
> Hauptstraße 47-51
> D-69117 Heidelberg Germany
>
> Phone: +49 6221 54 7746
> Mail:anna-lena.schubert using psychologie.uni-heidelberg.de
> Web:http://www.psychologie.uni-heidelberg.de/ae/diff/diff/people-schubert.html
>
>
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
--
Michael
http://www.dewey.myzen.co.uk/home.html
More information about the R-sig-meta-analysis
mailing list