[R-meta] HKSJ adjusted error by hand

[Viechtbauer, Wolfgang (NP)]

I have played around with this idea and this is even implemented in rma.mv():

dat <- dat.konstantopoulos2011
rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, test="knha")

As you will see in the warning message, this is experimental. And based on some simulations I have done, this doesn't actually work that great. Maybe this could be improved by using a Satterthwaite approximation to the dfs. But even then, I am not sure if this will do all that much better than:

rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, test="t", dfs="contain")

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Yefeng Yang via R-sig-meta-analysis
> Sent: Thursday, September 26, 2024 17:35
> To: James Pustejovsky <jepusto using gmail.com>; R Special Interest Group for Meta-
> Analysis <r-sig-meta-analysis using r-project.org>
> Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> Subject: Re: [R-meta] HKSJ adjusted error by hand
>
> Hi James,
>
> Thank you for you quick and helpful code.
>
> Interesting to see the scale factor (S_sq  in you notation) is just the
> heterogeneity measure H^2, which is equal to Q / df. If this is the case, I am
> wondering why this adjustment has not been extended to multilevel/multivariate
> meta-analysis, given that one can also derive the scale factor from these
> relative complex meta-analyses?
>
> Best,
> Yefeng
>
>
> ________________________________
> From: James Pustejovsky <jepusto using gmail.com>
> Sent: Friday, September 27, 2024 0:41
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> Subject: Re: [R-meta] HKSJ adjusted error by hand
>
> The adjustment is just a scalar that can be calculated "by hand" from the
> weights and residuals of the model. Here's an example, which could be turned
> into a function:
>
> library(metafor)
>
> dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
>
> rma_fit <- rma(yi, vi, mods = ~ ablat + year, data=dat)
>
> # Calculate variance factor
> wi <- 1 / (rma_fit$vi + rma_fit$tau2)
> ri <- residuals(rma_fit)
> df <- rma_fit$k - rma_fit$p
> S_sq <- sum(wi * ri^2) / df
>
> # Scale the variance-covariance matrix
> V_hksj <- S_sq * vcov(rma_fit)
>
> # Check against rma.uni()
> rma_hksj <- update(rma_fit, test = "hksj")
> all.equal(V_hksj, vcov(rma_hksj))
>
> Depending on your preferred settings, you might need to adjust by taking
> max(S_sq, 1).
>
> James
>
> On Thu, Sep 26, 2024 at 9:03 AM Yefeng Yang via R-sig-meta-analysis <r-sig-meta-
> analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>> wrote:
> Dear community,
>
> I am wondering whether there is a post-hoc way to calculate sampling variances
> of the estimated regression coefficients from meta-analysis models based on the
> Hartung-Knapp-Sidik-Jonkman method.
>
> To be more precise,
>
>   1.
> I first fit a normal random-effect meta-analysis via `rma()` in metafor package:
>
> library(metafor)
> dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
> mod <- rma(yi, vi, data=dat),
>
>
>   1.
> then, how can I get the adjusted standard error of the estimated coefficients
> (in this case, it is an intercept) based on the model object `mod`?
>
> Of course, we can get it using an on-the-fly way:
> rma(yi, vi, data=dat, test = "hksj")
>
> But I have a couple of big datasets and want to report both the original
> standard and adjusted errors. I do not have a high-performance PC and I would
> like to avoid 're-fitting' the meta-analysis.
>
> Best,
> Yefeng
>
>
>
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org<mailto:R-
> sig-meta-analysis using r-project.org>
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
>       [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis