[R-meta] Meta - Bug with REML or small N?

[Jorge Teixeira]

So, did I just win the lottery? 😅

Would you recommend switching to another estimator? DL maybe?

I mean, the t2 value can be "correct", but does not look appropriate when
you look at the forest plot.




Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
escreveu no dia terça, 21/03/2023 à(s) 17:13:

> So the REML estimate is correct:
>
> dat <- structure(list(study = c("AA", "BB", "CC",
> "EE", "DDD"), en = c(9, 41, 29, 8, 13
> ), em = c(32, 27.5, 28.7, 22.8, 30.5), esd = c(1.9, 3.8, 5.2,
> 3.8, 4.9), cn = c(8, 26, 28, 10, 14), cm = c(30.1, 24.9, 26.9,
> 24.7, 30.2), csd = c(2.4, 3.6, 3, 4.3, 5.1)), row.names = c(NA,
> -5L), class = c("tbl_df", "tbl", "data.frame"))
>
> dat <- escalc(measure="MD", m1i=em, sd1i=esd, n1i=en, m2i=cm, sd2i=csd,
> n2i=cn, data=dat)
>
> res <- rma(dat)
> res
> profile(res, steps=100)
>
> Rather different tau^2 estimates depending on the estimator:
>
> methods <- c("ML","REML","EB","PM","HS","HSk","HE","DL","SJ")
> data.frame(tau2=sapply(methods, \(x) round(rma(dat, method=x)$tau2, 4)))
>
> Sometimes life is difficult.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Jorge Teixeira via R-sig-meta-analysis
> >Sent: Tuesday, 21 March, 2023 17:56
> >To: Michael Dewey
> >Cc: Jorge Teixeira; R Special Interest Group for Meta-Analysis
> >Subject: Re: [R-meta] Meta - Bug with REML or small N?
> >
> >Thanks Michael and Wolfgang.
> >
> >Here goes the df:
> >
> >structure(list(study = c("AA", "BB", "CC",
> >"EE", "DDD"), en = c(9, 41, 29, 8, 13
> >), em = c(32, 27.5, 28.7, 22.8, 30.5), esd = c(1.9, 3.8, 5.2,
> >3.8, 4.9), cn = c(8, 26, 28, 10, 14), cm = c(30.1, 24.9, 26.9,
> >24.7, 30.2), csd = c(2.4, 3.6, 3, 4.3, 5.1)), row.names = c(NA,
> >-5L), class = c("tbl_df", "tbl", "data.frame"))
> >
> >Thanks
> >
> >Michael Dewey <lists using dewey.myzen.co.uk> escreveu no dia terça, 21/03/2023
> >à(s) 16:36:
> >
> >> Dear Jorge
> >>
> >> It looks as though the issue is that REML estimates tau^2 to be
> >> effectively zero whereas DL has a positive estimate. The different ways
> >> of estimating tau^2 do often differ, sometimes by quite surprising
> >> amounts. Whether that is related to the smallish number of studies I
> >> could not say but, at the moment, there does not seem to be evicence of
> >> anything wrong with the implementation of REML.
> >>
> >> Michael
> >>
> >> On 21/03/2023 14:57, Jorge Teixeira via R-sig-meta-analysis wrote:
> >> > Hi.
> >> >
> >> > Let me know if you need me to provide the data for this example.
> >> > Screenshots in the bottom.
> >> >
> >> > I ran this MA with REML, and the weight for random and common effects
> >> > are exactly the same! Never saw anything like this. t2values also
> don’t
> >> > look plausible.
> >> >
> >> > *_#1_*
> >> > vo2 <- metacont(en  , em, esd, cn, cm, csd, study, method.tau =
> "REML",
> >> > prediction = TRUE, data = dat_vo2, sm = "MD")
> >> > vo2
> >> >
> >> > Is this a bug or a particular issue of low number of studies and low
> >> > sample size?
> >> >
> >> > *_#2_*
> >> > vo2 <- metacont(en  , em, esd, cn, cm, csd, study, method.tau = "DL",
> >> > prediction = TRUE, data = dat_vo2, sm = "MD")
> >> > vo2
> >> >
> >> > I ran this with DL estimator and weights and t2are plausible. I also
> ran
> >> > other similar MA using REML and this was all okay.
> >> >
> >> > #1 using SMD instead of MD also looks fine.
> >> >
> >> >
> >> > Thanks,
> >> > Jorge
> >> >
> >> >
> >> > *_REML:_*
> >> >
> >> > image.png
> >> >
> >> >
> >> > *_DL:_*
> >> > image.png
>

	[[alternative HTML version deleted]]