[R-meta] Comparing dependent, overlapping correlation coefficients
Anna-Lena Schubert
@nn@-len@@@chubert @ending from p@ychologie@uni-heidelberg@de
Fri Aug 17 14:50:44 CEST 2018
Dear Michael,
but how do I test if the effect of factor levels X vs Y (leaving out Z)
is moderated by my continuous covariate? I believe that metafor gives me
several ways to test for an overall interaction between a factor and a
continuous covariate, but I'm not sure how I'd do follow-up comparisons
from there.
Best,
Anna-Lena
Am 17.08.2018 um 14:44 schrieb Michael Dewey:
> Dear Anna-Lena
>
> The concept of interaction works generally for factor by factor,
> factor by covariate, and covariate by covariate. So you can just go
> ahead. I must say I have always found it harder to explain the
> covariate by covariate ones but that may be a defect in my explanatory
> powers.
>
> Michael
>
> On 17/08/2018 12:43, Anna-Lena Schubert wrote:
>> Dear Wolfgang,
>>
>> thank you so much, this works perfectly well for me!
>>
>> I have one final questions before I'm ready to analyze my data: Could
>> I check whether this moderation by variable type is moderated by
>> study characteristics? I.e., is there a way to include an interaction
>> term that again specifically tests if moderator M moderates the
>> difference in correlations between X and Y? I found your example on
>> two categorial moderates and think I could apply that, but most of
>> the moderators I'm thinking about are metric variables.
>>
>> Best,
>>
>> Anna-Lena
>>
>>
>> Am 15.08.2018 um 14:50 schrieb Viechtbauer, Wolfgang (SP):
>>> library(metafor)
>>>
>>> source("https://gist.githubusercontent.com/wviechtb/700983ab0bde94bed7c645fce770f8e9/raw/5bb5601852b132af533aef41405d58a3ae04cf82/rmat.r")
>>>
>>>
>>> dat <- read.table(header=TRUE, text = "
>>> study var1 var2 ri ni
>>> 1 X Y .20 50
>>> 1 X Z .30 50
>>> 1 Y Z .52 50
>>> 2 X Y .34 35
>>> 2 X Z .43 35
>>> 2 Y Z .44 35")
>>>
>>> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat)
>>> dat2
>>>
>>> res <- rma.mv(yi, dat2$V, mods = ~ var1var2 - 1, random = ~ var1var2
>>> | id, struct="UN", data=dat2$dat)
>>> res
>>>
>>> ### three contrasts
>>> anova(res, L=c(1, -1, 0))
>>> anova(res, L=c(1, 0, -1))
>>> anova(res, L=c(0, 1, -1))
>>>
>>> You are interested in the first of these three contrasts, that is,
>>> whether cor(X,Y) = cor(X,Z).
>>>
>>> Actually, I would recommend to work with:
>>>
>>> dat2 <- rmat(ri ~ var1 + var2 | study, n=c(50,35), data=dat, rtoz=TRUE)
>>> dat2
>>>
>>> This applies Fisher's r-to-z transformation.
>>>
>>> Best,
>>> Wolfgang
>>>
>>> -----Original Message-----
>>> From: Anna-Lena Schubert
>>> [mailto:anna-lena.schubert using psychologie.uni-heidelberg.de]
>>> Sent: Wednesday, 15 August, 2018 11:08
>>> To: Viechtbauer, Wolfgang (SP);r-sig-meta-analysis using r-project.org
>>> Subject: Re: [R-meta] Comparing dependent, overlapping correlation
>>> coefficients
>>>
>>> Hi Wolfgang,
>>>
>>> thanks so much, I now believe I have sensible values in my V matrix.
>>> They deviate slightly from manual calculations, but are really close.
>>>
>>> I'm still lost on how to test for the interaction between X and Y then,
>>> though. I tried
>>>
>>> res <- rma.mv(yi, V, mods = ~ variable | studyID, data=dat,
>>> method="ML"),
>>>
>>> but the results don't correspond to the univariate meta-analyses I
>>> conducted before. In addition, it tests (I believe) for a moderation of
>>> all three correlations, while I'm only interested in the difference
>>> between r_XY and r_XZ. Moreover, the manual says "In case the sampling
>>> errors are correlated, then one can specify the entire
>>> variance-covariance matrix of the sampling errors via the V argument",
>>> but it seems I cannot simply leave "yi" out of the argument.
>>>
>>> Best, Anna-Lena
>>>
>>> Am 14.08.2018 um 22:18 schrieb Viechtbauer, Wolfgang (SP):
>>>> You do not need escalc(). The rmat() function gives you the
>>>> variances along the diagonal of the 'V' matrix.
>>>>
>>>> The variances should be (1 - ri^2)^2 / (ni - 1). You should be able
>>>> to double-check that these values correspond to your data. Since ni
>>>> should be the same for r_XY and r_XZ within a study, then it might
>>>> be that the variances are roughly the same if the two correlations
>>>> are not all that different. They should not be identical though
>>>> (unless r_XY and r_XZ are the same).
>>>> Best,
>>>> Wolfgang
>>>>
>>>> -----Original Message-----
>>>> From: R-sig-meta-analysis
>>>> [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of
>>>> Anna-Lena Schubert
>>>> Sent: Tuesday, 14 August, 2018 14:20
>>>> To: James Pustejovsky
>>>> Cc:r-sig-meta-analysis using r-project.org
>>>> Subject: Re: [R-meta] Comparing dependent, overlapping correlation
>>>> coefficients
>>>>
>>>> Hi James,
>>>> I used Wolfgang's script on git to calculate the Cov(r_XY, r_XZ) by
>>>> feeding it Cor(r_YZ). In the next step, I calculated Var(r_XY) and
>>>> Var(r_XZ) by using the escalc function. However, Var(r_XY) always
>>>> equals Var(r_XZ) for each study. Does this make sense?
>>>> I nevertheless added all three measures per study into a
>>>> variance-covariance matrix such as:
>>>> r_XY r_XZ r_XY r_XZ
>>>> r_XY 0.004 0.0001 0 0
>>>> r_XZ 0.0001 0.004 0 0
>>>> r_XY 0 0 0.008 0.002
>>>> r_XZ 0 0 0.002 0.008
>>>> Then, I tried to feed everything into a multivariate meta-analysis:
>>>> res <- rma.mv(yi, V, mods = ~ variableType - 1, random = ~
>>>> variableType | studyNum, struct="UN", data=dat, method="ML")
>>>> The estimates I get for both of the correlation coefficients
>>>> correspond closely to those I get when only meta-analyzing one of
>>>> the variable types, which seems great. However, I'm still somewhat
>>>> concerned that Var(r_XY) = Var(r_XZ). Do you think there may have
>>>> been some mistake in my code or does it make sense that these
>>>> variances are equal?
>>>> Best,
>>>> Anna-Lena
>>>> Am 10.08.2018 um 17:06 schrieb James Pustejovsky:
>>>> Anna-Lena,
>>>>
>>>> The approach that you suggested (putting the data in "long" format
>>>> and defining an indicator variable for whether Y or Z is the
>>>> correlate) is just what I would recommend. However, there is a
>>>> complication in that the estimates r_XY and r_XZ are correlated
>>>> (correlated correlation coefficients...say that six times fast!),
>>>> and the degree of correlation depends on r_YZ.
>>>>
>>>> 1) If you have extracted data on r_YZ then you could use this to
>>>> compute Cov(r_XY, r_XZ) and then do a multivariate meta-analysis.
>>>> See discussion here:
>>>> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-January/000483.html
>>>>
>>>> And this function for computing the required covariance matrices:
>>>> https://gist.github.com/wviechtb/700983ab0bde94bed7c645fce770f8e9
>>>> There are at least three further alternatives that might be simpler:
>>>>
>>>> 2) If you have r_YZ you could use it to compute the sampling
>>>> variance of the difference between r_XY and r_XZ, that is:
>>>>
>>>> Var(r_XY - r_XZ) = Var(r_XY) + Var(r_XZ) - 2 * Cov(r_XY, r_XZ)
>>>>
>>>> You could then do a univariate meta-analysis on the difference
>>>> between correlations.
>>>>
>>>> 3) If you do not have r_YZ then you won't be able to estimate
>>>> Cov(r_XY, r_XZ) very well. You could make a guess about r_YZ and
>>>> then follow approach (1) or (2) above, using cluster-robust
>>>> variance estimation to account for the possibly mis-estimated
>>>> sampling-variance covariance matrix.
>>>>
>>>> 4) Or you could ignore the covariance between r_XY and r_XZ
>>>> entirely, fit the model to the long data as you describe above, and
>>>> use cluster-robust variance estimation (clustering by sample) to
>>>> account for the dependence between r_XY and r_XZ. This is the
>>>> quickest and dirtiest approach, and the first thing I would try in
>>>> practice before moving on to the more refined approaches above.
>>>>
>>>> James
>>>> On Fri, Aug 10, 2018 at 9:21 AM Anna-Lena
>>>> Schubert<anna-lena.schubert using psychologie.uni-heidelberg.de> wrote:
>>>> Dear all,
>>>>
>>>> I want to run a meta-analysis that compares dependent, overlapping
>>>> correlation coefficients (i.e., I want to see if X correlates more
>>>> strongly with Y than it does with Z). I already ran a meta-analysis
>>>> separately for both of these correlations and would now like to
>>>> compare
>>>> those two pooled effect sizes statistically. Confidence intervals
>>>> of the
>>>> two correlations do not overlap (r1 = .18 [.12; .24]; r2 = .32 [.25;
>>>> .39]), but I wonder if there may be a more elegant way to compare
>>>> these
>>>> correlations than just based on CIs.
>>>>
>>>> I wonder, for example, if a factorial variable could be used to
>>>> identify
>>>> those correlations in a "long" data format style, and if I could test
>>>> for a significant interaction between variable type (Y vs. Z) and the
>>>> correlation in a meta-analysis:
>>>>
>>>> Study Variable r
>>>> 1 Y .20
>>>> 1 Z .30
>>>> 2 Y .34
>>>> 2 Z .43
>>>>
>>>> I would greatly appreciate if anyone could tell me if that's a good
>>>> idea
>>>> or could recommend other approaches. Thanks in advance for any
>>>> offers of
>>>> help!
>>>>
>>>> Best,
>>>> Anna-Lena
>>
>> --
>> Signatur
>>
>>
>> Dr. Anna-Lena Schubert
>>
>> Postdoc at Section of Personality
>> Heidelberg University - Institute of Psychology
>>
>> Hauptstraße 47-51
>> D-69117 Heidelberg Germany
>>
>> Phone: +49 6221 54 7746
>> Mail:anna-lena.schubert using psychologie.uni-heidelberg.de
>> Web:http://www.psychologie.uni-heidelberg.de/ae/diff/diff/people-schubert.html
>>
>>
>>
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>
--
Signatur
Dr. Anna-Lena Schubert
Postdoc at Section of Personality
Heidelberg University - Institute of Psychology
Hauptstraße 47-51
D-69117 Heidelberg Germany
Phone: +49 6221 54 7746
Mail: anna-lena.schubert using psychologie.uni-heidelberg.de
Web: http://www.psychologie.uni-heidelberg.de/ae/diff/diff/people-schubert.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180817/96f80b1b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fapahjabhiekfcgi.png
Type: image/png
Size: 25980 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180817/96f80b1b/attachment-0001.png>
More information about the R-sig-meta-analysis
mailing list