[R-meta] Why does rma.mv does not show the same results as robumeta?

[Viechtbauer, Wolfgang (SP)]

Dear Cátia,

robumeta uses robust variance estimation. If you want to do the same based on an 'rma.mv' object, you need to use robust() or, even better, the clubSandwich package. See here for examples:

https://wviechtb.github.io/metafor/reference/robust.html

However, the results still won't be exactly the same. There is at least one post in the archives that discusses the somewhat subtle differences. If you go here:

https://www.google.com/search?hl=EN&source=hp&q=site:https://stat.ethz.ch/pipermail/r-sig-meta-analysis

you can add some appropriate search strings to find those posts (I believe it was James Pustejovksy that explained this quite thoroughly, so you might want to include 'James' in your search terms).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Cátia Ferreira De Oliveira
>Sent: Sunday, 23 May, 2021 3:51
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Why does rma.mv does not show the same results as robumeta?
>
>Hello,
>
>I have conducted a meta-analysis that I am currently analysing looking at the
>relationship between memory and language/literacy and multiple studies contributed
>more than one effect size. I have preregistered doing the analyses in robumeta.
>But I am interested in checking how the results converge across packages as I am
>tempted to use metafor for my next meta-analysis given how easy it is to plot,
>check for publication bias, etc with this package. When running both models, they
>produced different results and I am a bit unsure as to why they are different. I
>know if I look at the estimates it is not that different, but what surprises me is
>the fact that DD has a higher estimate in one model but in the other it is the DLD
>group. Maybe I have done something wrong. Does anyone have any thoughts?
>
># multilevel model looking at the relationship between memory and
>language/literacy;
># multiple studies have contributed multiple effect sizes
>
>head(Data)
>
>rma.model <- rma.mv(yi, vi,  mods =  ~ factor(Group)-1,  random= ~ 1 |
>Study/effectsizeID, data=Data)
>res
>
>Multivariate Meta-Analysis Model (k = 414; method: REML)
>
>  logLik  Deviance       AIC       BIC      AICc
>-13.0662   26.1323   36.1323   56.2253   36.2805
>
>Variance Components:
>
>            estim    sqrt  nlvls  fixed              factor
>sigma^2.1  0.0109  0.1044     37     no               Study
>sigma^2.2  0.0082  0.0903    414     no  Study/effectsizeID
>
>Test for Residual Heterogeneity:
>QE(df = 411) = 588.9613, p-val < .0001
>
>Test of Moderators (coefficients 1:3):
>QM(df = 3) = 11.1370, p-val = 0.0110
>
>Model Results:
>
>robu.model <- robu(formula = yi ~ factor(Group)-1, data = Data,
>                       studynum = Study, var.eff.size = vi,
>                       rho = .8, small = TRUE)
>print(robu.model)
>
>RVE: Correlated Effects Model with Small-Sample Corrections
>
>Model: yi ~ factor(Group) - 1
>
>Number of studies = 37
>Number of outcomes = 414 (min = 1 , mean = 11.2 , median = 6 , max = 52 )
>Rho = 0.8
>I.sq = 52.35398
>Tau.sq = 0.02918897
>
>Thank you!
>
>Best wishes,
>
>Catia