[R-meta] multilevel meta-analysis using metafor

[Wolfgang Viechtbauer]

Please keep the mailing list in cc.

Yes, this means the subjects overlap, that is, the correlations are 
computed based on the same sample. In that case, the correlations are 
correlated. Equations for computing the covariances can be found in:

Steiger, J. H. (1980). Tests for comparing elements of a correlation 
matrix. Psychological Bulletin, 87(2), 245-251.

There are various cases. Let's say there are four variables: x1, x2, x4, 
and x4, all measured in the same sample. Then we have the case of 
non-overlapping variables:

cov(cor(x1,x2), cor(x3,x4))

To compute that covariance, you will need the full 4x4 correlation matrix.

And there is the case of partially overlapping variables, for example:

cov(cor(x1,x2), cor(x1,x3))

To compute the covariance here, you will need cor(x2,x3) (obviously, 
cor(x1,x2) and cor(x1,x3) you already have, otherwise you would not be 
interested in their covariance).

Again, the necessary equations can be found in Steiger (1980).

If you do not have the information to compute the covariances, then we 
are back to the situation where the covariances between the outcomes 
cannot be computed. See previous posts on how to deal with that. For 
example:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000097.html
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000094.html

Best,
Wolfgang

-- 
Wolfgang Viechtbauer, Ph.D., Statistician | Department of Psychiatry and
Neuropsychology | Maastricht University | P.O. Box 616 (VIJV1) | 6200 MD
Maastricht, The Netherlands | +31 (43) 388-4170 | http://www.wvbauer.com

On 08/30/2017 01:19 PM, brauldeq wrote:
Dear Wolfgang,

thanks for your fast reply. I do not quite understand what you mean by
"overlap in the subjects used to compute the various correlation
coefficients within the same study". Each of the multiple (dependent)
effect sizes within one particular study is based on the same
participants. For example, one study would report multiple correlations
for the relationship between scholastic achievement (i.e. GPA) and
various self-efficacy measures (i.e. subject-specific self-efficacy,
general self-efficacy, self-efficacy for self-regulated learning, etc.)
that were all based on the same set of participants.
Would that mean an overlap in subjects? Or would I be able to use
rma.mv(yi, vi, random = ~ 1 | sample_nr/effect_nr, data = se_ach)?

Thanks for your advice.

Best regards,
Denise

Am 30.08.2017 13:07, schrieb Wolfgang Viechtbauer:
Yes, that would be appropriate/sufficient, assuming there is no
overlap in the subjects used to compute the various correlation
coefficients within the same study. But I suspect that may not be the
case. If so, the covariances of the correlations are not 0 (as is
assumed by the model below).

Best,
Wolfgang

On 08/30/2017 11:54 AM, brauldeq wrote:
Dear fellow researchers,

I am currently conducting a meta-analysis for my master's thesis in
psychology. For that matter, I stumbled upon your metafor package which
has been very helpful for me!

However, I have some trouble implementing the right R code for my
analysis. I am running a meta-analysis on the relationship between
self-efficacy and scholastic achievement (using correlation
coefficients). It will be analyzed using multilevel analysis using a
random-effects model because I have several articles with multiple
effect sizes which I want to include individually controlling for their
dependency. Therefore, I have (dependant) effect sizes that are nested
within (independent) samples.
Each independent sample is coding with an unique number in the variable
"sample_nr" and each dependant effect size for a certain sample is coded
with an unique number in "effect_nr". So, a sample with multiple effect
sizes would be coded with an identical "sample_nr" and different
"effect_nr" for each effect size. My R code is as follows:

rma.mv(yi, vi, random = ~ 1 | sample_nr/effect_nr, data = se_ach)

Is this the right code for my purpose?

Thank you for your advice.

Best regards,
Denise Braul