[R] meta analysis with repeated measure-designs?
Mike Cheung
mikewlcheung at gmail.com
Wed Jun 16 10:47:29 CEST 2010
Dear Gerrit,
If the correlations of the dependent effect sizes are unknown, one
approach is to conduct the meta-analysis by assuming that the effect
sizes are independent. A robust standard error is then calculated to
adjust for the dependence. You may refer to Hedges et. al., (2010) for
more information. I have coded it here for reference.
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance
estimation in meta-regression with dependent effect size estimates.
Research Synthesis Methods, 1(1), 39-65. doi:10.1002/jrsm.5
Regards,
Mike
--
---------------------------------------------------------------------
Mike W.L. Cheung Phone: (65) 6516-3702
Department of Psychology Fax: (65) 6773-1843
National University of Singapore
http://courses.nus.edu.sg/course/psycwlm/internet/
---------------------------------------------------------------------
library(metafor)
#### Robust SE based on Hedges et al., (2010) Eq. 6
#### rma.obj: object fitted by metafor()
#### cluster: indicator for clusters of studies
robustSE <- function(rma.obj, cluster=NULL, CI=.95) {
# m: no. of clusters; assumed independent if not specified
if (is.null(cluster)) {
m=nrow(rma.obj$X)
} else {
m=nlevels(unique(as.factor(cluster)))
}
res2 <- diag(residuals(rma.obj)^2)
X <- rma.obj$X
b <- rma.obj$b
W <- diag(1/(rma.obj$vi+rma.obj$tau2)) # Use vi+tau2
meat <- t(X) %*% W %*% res2 %*% W %*% X # W is symmetric
bread <- solve( t(X) %*% W %*% X)
V.R <- bread %*% meat %*% bread # Robust sampling covariance matrix
p <- length(b) # no. of predictors
including intercept
se.R <- sqrt( diag(V.R)*m/(m-p) ) # small sample adjustment (Eq.7)
t.R <- b/se.R
p.R <- 2*(1-pt(abs(t.R),df=(m-p)))
crit <- qt( 1-CI/2, df=(m-p) )
ci.lb <- b-crit*se.R
ci.ub <- b+crit*se.R
data.frame(estimate=b, se.R=se.R, t.R=t.R, p.R=p.R, ci.lb=ci.lb, ci.ub=ci.ub)
}
## Example: calculate log relative risks and corresponding sampling variances
data(dat.bcg)
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
dat <- cbind(dat.bcg, dat)
## random-effects model
fit1 <- rma(yi, vi, data=dat, method="DL")
summary(fit1)
estimate se zval pval ci.lb ci.ub
-0.7141 0.1787 -3.9952 <.0001 -1.0644 -0.3638 ***
## Robust SE
robustSE(fit1)
estimate se.R t.R p.R ci.lb ci.ub
intrcpt -0.7141172 0.1791445 -3.986265 0.001805797 -0.725588 -0.7026465
## mixed-effects model with two moderators (absolute latitude and
publication year)
fit2 <- rma(yi, vi, mods=cbind(ablat, year), data=dat, method="DL")
summary(fit2)
estimate se zval pval ci.lb ci.ub
intrcpt -1.2798 25.7550 -0.0497 0.9604 -51.7586 49.1990
ablat -0.0288 0.0090 -3.2035 0.0014 -0.0464 -0.0112 **
year 0.0008 0.0130 0.0594 0.9526 -0.0247 0.0262
### Robust SE
robustSE(fit2)
estimate se.R t.R p.R ci.lb
intrcpt -1.2797914381 22.860353022 -0.05598301 0.956458098 -2.749670e+00
ablat -0.0287644840 0.007212163 -3.98832970 0.002566210 -2.922821e-02
year 0.0007720798 0.011550188 0.06684565 0.948022174 2.942412e-05
ci.ub
intrcpt 0.190086881
ablat -0.028300755
year 0.001514735
On Mon, Jun 14, 2010 at 4:58 PM, Gerrit Hirschfeld
<Gerrit.Hirschfeld at uni-muenster.de> wrote:
> Hi,
>
> thanks for the references I will try the sensitivitiy-analysis in R and try out winbugs if that does not work (little afraid of switching programmes).
>
> I also had an idea for a reasonable estimate of the correlations. Some studies report both results from paired t-tests and means and SDs, and thus allow to calculate two estimates for d one based on M and SD alone the other on t. The difference between the two estimates should be systematically related to the correlations of measures.
>
> I will keep you posted, if I have a solution or hit a wall.
>
> efachristo and dank je wel!
>
> Gerrit
>
>
> On 12.06.2010, at 15:59, Viechtbauer Wolfgang (STAT) wrote:
>
>> Dear Gerrit,
>>
>> the most appropriate approach for data of this type would be a proper multivariate meta-analytic model (along the lines of Kalaian & Raudenbush, 1996). Since you do not know the correlations of the reaction time measurements across conditions for the within-subject designs, a simple solution is to "guestimate" those correlations and then conduct sensitivity analyses to make sure your conclusions do not depend on those guestimates.
>>
>> Best,
>>
>> --
>> Wolfgang Viechtbauer http://www.wvbauer.com/
>> Department of Methodology and Statistics Tel: +31 (0)43 388-2277
>> School for Public Health and Primary Care Office Location:
>> Maastricht University, P.O. Box 616 Room B2.01 (second floor)
>> 6200 MD Maastricht, The Netherlands Debyeplein 1 (Randwyck)
>>
>>
>> ----Original Message----
>> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On
>> Behalf Of Gerrit Hirschfeld Sent: Saturday, June 12, 2010 12:45
>> To: r-help at r-project.org
>> Subject: [R] meta analysis with repeated measure-designs?
>>
>>> Dear all,
>>>
>>> I am trying to run a meta analysis of psycholinguistic reaction-time
>>> experiments with the meta package. The problem is that most of the
>>> studies have a within-subject designs and use repeated measures ANOVAs to
>>> analyze their data. So at present it seems that there are three
>>> non-optimal ways to run the analysis.
>>>
>>> 1. Using metacont() to estimate effect sizes and standard errors. But as
>>> the different sores are dependent this would result in biased estimators
>>> (Dunlap, 1996). Suppose I had the correlations of the measures (which I
>>> do not) would there by an option to use them in metacont() ?
>>>
>>> 2. Use metagen() with an effect size that is based on the reported F for
>>> the contrasts but has other disadvantages (Bakeman, 2005). The problem I
>>> am having with this is that I could not find a formular to compute the
>>> standard error of partial eta squared. Any Ideas?
>>>
>>> 3. Use metagen() with r computed from p-values (Rosenthal, 1994) as
>>> effect size with the problem that sample-size affects p as much as effect
>>> size.
>>>
>>> Is there a fourth way, or data showing that correlations can be neglected
>>> as long as they are assumed to be similar in the studies?
>>> Any ideas are much apprecciated.
>>>
>>> best regards
>>> Gerrit
>>>
>>> ______________________________
>>> Gerrit Hirschfeld, Dipl.-Psych.
>>>
>>> Psychologisches Institut II
>>> Westfälische Wilhelms-Universität
>>> Fliednerstr. 21
>>> 48149 Münster
>>> Germany
>>>
>>> psycholinguistics.uni-muenster.de
>>> GerritHirschfeld.de
>>> Fon.: +49 (0) 251 83-31378
>>> Fon.: +49 (0) 234 7960728
>>> Fax.: +49 (0) 251 83-34104
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________
> Gerrit Hirschfeld, Dipl.-Psych.
>
> Psychologisches Institut II
> Westfälische Wilhelms-Universität
> Fliednerstr. 21
> 48149 Münster
> Germany
>
> psycholinguistics.uni-muenster.de
> GerritHirschfeld.de
> Fon.: +49 (0) 251 83-31378
> Fon.: +49 (0) 234 7960728
> Fax.: +49 (0) 251 83-34104
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
More information about the R-help
mailing list