[R-meta] Egger-type test for multi-level meta-analysis

[Viechtbauer, Wolfgang (NP)]

Just me thinking out loud here for the moment:

While the extension of the 'Egger regression test' (and the PET/PEESE methods) to multilevel models is straightforward, a bit of thinking is required as to what we are really trying to capture by adding something like sqrt(vi) (or just vi or any other transformation thereof) to the model as a predictor. Selective reporting within studies of the larger / significant effects? Or selective reporting of studies in general? In the latter case, selection may depends on whether at least one effect is significant / large, the focal one (in case there is a defined primary endpoint), or something else. One could argue that cor(sqrt(vi), yi) (in essence what we are examining with the regression test) might capture a bit of all of this and maybe that's the best we can generally do.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of James Pustejovsky via R-sig-meta-analysis
>Sent: Monday, 19 June, 2023 18:23
>To: R Special Interest Group for Meta-Analysis
>Cc: James Pustejovsky
>Subject: Re: [R-meta] Egger-type test for multi-level meta-analysis
>
>Correct. The slope on sqrt(vi) indicates an association between SE and
>effect size. The intercept is a "PET"-style estimate of the average effect
>size in a population of infinitely large studies (i.e. SE = 0).
>
>James
>
>On Mon, Jun 19, 2023 at 9:21 AM Dr. Guido Schwarzer via R-sig-meta-analysis
><r-sig-meta-analysis using r-project.org> wrote:
>
>> Hi all,
>>
>> Another question on multi-level models (while I am still waiting for an
>> answer on my previous one ;-) ).
>>
>> I would like to conduct a test for small-study effects for data from a
>> three-level model, e.g., for the dataset dat.konstantopoulos2011.
>>
>> library("metafor")
>> library("metadat")
>> m.ml <- rma.mv(yi, vi, random = ~ 1 | district / school, data =
>> dat.konstantopoulos2011)
>>
>> If I understand James' comment from February 2018 correctly (
>> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-February/000610.html),
>> I could conduct an Egger-type test for small study effects by using
>> cluster-robust variance estimation following a multi-level meta-regression
>> with the standard error as moderator:
>>
>> sse.ml <- update(m.ml, mods = sqrt(vi))
>> library("clubSandwich")
>> conf_int(sse.ml, vcov = "CR2")
>>
>> Did I get this right?
>>
>> Best wishes,
>> Guido