[R-meta] The role of 'W' argument in the rma.mv function

[Viechtbauer, Wolfgang (SP)]

Hi Gabri,

Let's start with rma() (i.e., rma.uni()). A random-effects model fitted with this function assumes:

yi ~ N(mu, tau^2 + vi),

where tau^2 is the estimated amount of heterogeneity (i.e., variance in the true effects) and vi is the sampling variance of the ith study. So, the 'tau^2 + vi' values are the *marginal variances* of the estimates implied by the model. To keep things simple, I am just going to consider models without moderators and hence there is just the fixed effect 'mu' that we would like to estimate.

Under this model, the most efficient estimator of mu is given by

mu-hat = sum(wi * yi) / sum(wi)

where wi = 1/(tau^2 + vi), that is, the inverse of the marginal variances.

In some cases, we might actually want to deviate from using these weights. For example, suppose we are worried about publication bias. Publication bias will tend to bias smaller studies more than larger studies. If tau^2 is large, then mu-hat is almost like an unweighted average and smaller and larger studies receive almost the same weights in computing mu-hat. This can lead to substantial bias in mu-hat. Instead, we can estimate mu with wi = 1/vi (what some would call "fixed-effects model" weights, but this is a confusing distraction, since we are still fitting a random-effects model), so that smaller studies receive little weight and larger studies more weight. As a result, mu-hat can have considerably less bias. See Henmi and Copas (2010).

To fit such a model, we would use:

rma(yi, vi, weights = 1/vi, method="REML", data=dat)

It is a common misconception to think that using 1/vi weights implies that we are fitting a fixed-effects model. The model fitted above is still a random-effects model, but with custom weights. The standard error (and hence CI for mu) will properly reflect the additional variability introduced by tau^2.

*** 

Sidenote: Henmi and Copas (2010) have also suggested a method along these lines that uses a more complex way of computing the SE and CI. This can be obtained with:

res <- rma(yi, vi, method="REML", data=dat) # i.e., fit model with standard weights
hc(res) # show 'standard' results and results from the H&C method

***

There may be other reasons why we might not want to use the standard weights. For example, for many effect size measures, vi is not only a function of the sample sizes, but also of the size the effect itself. This is not desirable. For some measures, this usually doesn't affects the results in a noteworthy manner (e.g., standardized mean differences), but it can for other measures (e.g., raw correlation coefficients). In that case, we might want to use just the sample sizes as weights. Here, we would use:

rma(yi, vi, weights = ni, method="REML", data=dat)

(assuming that variable 'ni' is in the dataset 'dat'). Note that one must still specify the sampling variances, as they are needed in correctly computing the SE (and hence CI).

Now let's consider rma.mv(), which can fit much more complex models. In very general terms, a model fitted by this function assumes

y ~ N(mu, V + D),

where y is the (column) vector with the effect size estimates, V is the var-cov matrix with the sampling variances along the diagonal and possibly covariances for the off-diagonal elements, and D is the var-cov matrix that is implied by the random effects added to the model. D will often not just be a diagonal matrix, but also have non-zero off-diagonal elements.

So, M = V + D is the marginal var-cov matrix of the estimates as implied by the model. Here, the most efficient estimator of mu is given by:

mu-hat = (X'WX)^(-1) X'Wy

where X is just a column vector with 1s and W = M^(-1) (i.e., the inverse of M). So, W is a *weight matrix* and is typically not just a diagonal matrix.

In rma.mv(), there is the 'W' argument, which allows the user to define custom weights -- and even an entire custom weight matrix (in fact, if a user just specifies a vector with custom weights, this is turned in a diagonal weight matrix).

As before, one might want to work with custom weights (or a custom weight matrix) in certain circumstances, for example to reduce the potentially biasing effects of publication bias or to avoid potential distortions as a result of a dependency between estimates and sampling variances. How best to do this in a given case for more complex models like those fitted by rma.mv() is not so clear.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Gabriele Midolo
Sent: Tuesday, 10 July, 2018 9:43
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] The role of 'W' argument in the rma.mv function

Dear all,

Could you provide an explanation or an example on why 'W' (for rma.mv) or
'weights' (for rma.uni) should be applied in a meta-analytic model? If I
understand correctly, one should in principle "weight" the observation by
the inverse of sampling variance. Is then possible to specify additional
"weight" to the observations in the model (assuming this is what 'W' does)?
If yes, when does it makes sense to use it?

Thanks,
Gabri