[R-meta] Standard error of the weighted mean in Metafor

[Viechtbauer Wolfgang (SP)]

If you are worried that the standard Wald-type test/CI (which is in essence what equation 6 gives you) does not provide valid inferences due to small k, then you can also use the Knapp and Hartung method (i.e, use argument test="knha"), which has repeatedly been shown to have pretty much nominal statistical properties (i.e., correct Type I error rate at the desired alpha level and correct coverage probability of the CI). The Knapp and Hartung method also can be used when fitting meta-regression models (as opposed to equation 7, whose generalization to models with moderators eludes me).

However, all of this is only relevant in the context of random/mixed-effects models. Loren, you mentioned fitting a fixed effect model. In that case, equation 6 is perfectly fine and there is no need for any adjustments.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of James Pustejovsky
Sent: Thursday, 07 September, 2017 4:21
To: Loren Albert; r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Standard error of the weighted mean in Metafor

Loren,

You are correct that the default standard error returned by rma.uni is
equivalent to Equation (6) of Hedges, Gurevitch, & Curtis (1999). As far as
I know, Equation (7) is not implemented in metafor. However, it is fairly
straight-forward to carry out the calculations in R. The code below gives
an example. I don't know whether Equation (7) really provides much
improvement in accuracy of SEs or CIs, but the difference is trivial in the
example given below. I'd be curious to hear if it makes any difference with
your data.

As an aside, Equation (7) appears to be generic, in the sense that it could
be applied to any effect size metric, rather than only to response ratios.
The example I give uses standardized mean differences.

James

# use first 8 studies from Raudenbush (1985)
data("dat.raudenbush1985")
dat <- subset(dat.raudenbush1985, study <= 10)

# random-effects model
reml_fit <- rma(yi, vi, data=dat, method="REML")
reml_fit

# calculate FE weights
w <- 1 / dat$vi

# calculate RE weights
w_star <- 1 / (dat$vi + reml_fit$tau2)

# Default metafor SE
reml_fit$se

# Formula (6) from HGC 1999 gives the same result
V_6 <- 1 / sum(w_star)
sqrt(V_6)

# Formula (7) from HGC 1999
df <- with(dat, n1i + n2i - 2)
theta <- w_star / sum(w_star)
V_7 <- (1 + 4 * sum(theta * (1 - theta) * (w_star / w)^2 / df)) /
sum(w_star)
sqrt(V_7)

# SE based on (7) compared to SE based on (6)
sqrt(V_7 / V_6)

On Wed, Sep 6, 2017 at 5:35 PM, Loren Albert <lalbert at email.arizona.edu>
wrote:

> Dear all,
>
> I am using the Metafor package (version 1.9-9) for a meta-analysis of log
> response ratios following Hedges et al. 1999 (citation at bottom). With
> escalc I use measure = "ROM" then I use rma to fit a fixed effect model (so
> method = "FE").
>
> Hedges et al give two different equations for the standard error of the
> weighted mean, one (equation 6) useful when the number of studies is large,
> and one (equation 7) useful when the sample size is smaller.
>
> In Metafor, I think that the standard error of the weighted mean produced
> from the 'rma' command is from equation 6, but I would like to verify this
> and also learn whether there is some option to use equation 7 instead.
> Overall I have found Metafor to be a great package with good documentation,
> so I apologize if I missed something about this in the documentation (so
> far I have looked at help(rma), the 'Metafor' document and Viechtbauer
> 2010).
>
> ---Reference---
> Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The Meta-Analysis of
> Response Ratios in Experimental Ecology. Ecology, 80(4), 1150–1156.
> http://doi.org/10.2307/177062?ref=search-gateway:
> d0e369adb64dd470ff732512b25b3dfd
>
> ----
> Thanks in advance!
>
> Loren