[R-meta] multilevel meta-analysis using metafor
brauldeq at hu-berlin.de
Mon Sep 11 14:32:23 CEST 2017
I am really sorry for the ongoing confusion on my part. But is there a
way to use weights corresponding to equation 13 or 17 from Hedges,
Tipton & Johnson (2010; see
http://onlinelibrary.wiley.com/doi/10.1002/jrsm.5/abstract) or to assign
equal weight to correlated within-study effect sizes?
I would like to avoid assigning more total weight to studies with a
larger number of (dependent) effect sizes than studies with fewer (or
only one) effect sizes.
Thanks a lot.
Am 06.09.2017 18:33, schrieb Viechtbauer Wolfgang (SP):
> As I just mentioned in another post, in models with more complex
> random effects structures, there are not just weights, but an entire
> weight matrix. For example, in the model you fitted below, the
> marginal var-cov matrix is block diagonal with vi + sigma^2_sample_nr
> + sigma^2_effect_nr along the diagonal and sigma^2_sample_nr for off
> diagonal elements that correspond to the same level of that factor.
> The weight matrix is then the inverse of that marginal var-cov matrix.
> If the sampling errors had been independent, then this would be the
> 'best' weight matrix (in the sense of giving you the most efficient
> estimates of the fixed effects and giving you appropriate estimates of
> the standard errors) and there would then be no need for any further
> adjustments, manually or statistically.
> In your case, the sampling errors are correlated, but you are fitting
> the model with V=vi, that is, assuming a diagonal V matrix. That will
> lead to less efficient but still unbiased estimates of the fixed
> effects, but invalid estimates of the standard errors, and hence the
> need for using robust(model, cluster=data$sample_nr). But this doesn't
> actually change the weight matrix; it only affects the way the var-cov
> matrix of the fixed effects is computed and hence the computation of
> the standard errors. So, also here there is no need to manually adjust
> any weights.
> And yes, *all else equal*, studies with more effect sizes are weighted
> more than studies with less effect sizes. Let me put this in a
> different way: A study providing more evidence about the phenomenon
> under investigation is getting more weight than a study providing less
> evidence. That seems pretty sensible to me.
> -----Original Message-----
> From: brauldeq [mailto:brauldeq at hu-berlin.de]
> Sent: Tuesday, 05 September, 2017 13:25
> To: Viechtbauer Wolfgang (SP)
> Cc: r-sig-meta-analysis at r-project.org
> Subject: Re: [R-meta] multilevel meta-analysis using metafor
> I was wondering how the weights are assigned correctly when specifying
> rma.mv(yi, vi, random = ~ 1 | sample_nr/effect_nr, data = data) and
> afterwards using
> robust.rma.mv(model, cluster=data$sample_nr, adjust=T) for a (cluster)
> robust estimation of variances and accurate assessment of standard
> Just a little reminder concerning my data structure. I have multiple
> effect sizes (all measured in the same sample) per study. The number of
> effect sizes within one study varies greatly from only 1 up until 30.
> I am asking because not long ago I was quite certain that by specifying
> a two-level random effects model, the dependency of effect sizes within
> studies would automatically be accounted for. Now I know that this is
> not the case and I need to account for the dependency of standard
> additionally with the robust() function. However, now I am wondering
> whether the weights are assigned correctly with the approach explained
> above. Or do I need to adjust the weights of studies with multiple
> (dependent) effect sizes manually? Specifically, do studies with more
> effect sizes are weighted more than studies with less effect sizes?
> would be very problematic in my case.
> Thanks for providing me with more details concerning the weighting of
> effect sizes within two-level random-effect models using the rma.mv()
> Best regards,
> Am 30.08.2017 16:31, schrieb Wolfgang Viechtbauer:
>> Yes, this is indeed how you can approach this.
>> And yes, if the var-cov structured is misspecified (which it is in
>> your case), then the fixed effects are still estimated unbiasedly
>> (although not as efficiently). The problem is that the SEs of the
>> fixed effects will not be correct. Using robust() allows you to get
>> more appropriate estimates of the SEs (and hence more appropriate
>> On 08/30/2017 03:10 PM, brauldeq wrote:
>> I understand that I could at first specify my model using model <-
>> rma.mv(yi, vi, random = ~ 1 | sample_nr/effect_nr, data = data). To
>> solve the problem concerning the covariances of the sampling errors I
>> would hereafter use robust.rma.mv(model, cluster=data$sample_nr,
>> = T). Would this approach solve my problem?
>> Am I right to assume that the rma.mv(yi, vi, random = ~1 |
>> sample_nr/effect_nr) function would calculate proper estimate of
>> size but is problematic in terms of sampling errors?
>> Am 30.08.2017 14:05, schrieb Wolfgang Viechtbauer:
>> Please keep the mailing list in cc.
>> Yes, this means the subjects overlap, that is, the correlations are
>> computed based on the same sample. In that case, the correlations are
>> correlated. Equations for computing the covariances can be found in:
>> Steiger, J. H. (1980). Tests for comparing elements of a correlation
>> matrix. Psychological Bulletin, 87(2), 245-251.
>> There are various cases. Let's say there are four variables: x1, x2,
>> x4, and x4, all measured in the same sample. Then we have the case of
>> non-overlapping variables:
>> cov(cor(x1,x2), cor(x3,x4))
>> To compute that covariance, you will need the full 4x4 correlation
>> And there is the case of partially overlapping variables, for example:
>> cov(cor(x1,x2), cor(x1,x3))
>> To compute the covariance here, you will need cor(x2,x3) (obviously,
>> cor(x1,x2) and cor(x1,x3) you already have, otherwise you would not be
>> interested in their covariance).
>> Again, the necessary equations can be found in Steiger (1980).
>> If you do not have the information to compute the covariances, then we
>> are back to the situation where the covariances between the outcomes
>> cannot be computed. See previous posts on how to deal with that. For
More information about the R-sig-meta-analysis