[R-meta] Publication bias with multivariate meta analysis

[Viechtbauer, Wolfgang (SP)]

Dear Huang,

Please find my comments below.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Huang Wu
>Sent: Saturday, 28 August, 2021 3:19
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Publication bias with multivariate meta analysis
>
>Hi all,
>
>I am conducting a multivariate meta-analysis using rma.mv. I want to test for
>publication bias.
>I noticed in a previous post, Dr. Pustejovsky provided the following code for
>Egger’s test.
>
>egger_multi <- rma.mv(yi = yi, V = sei^2, random = ~ 1 | studyID/effectID,
>mods = ~ sei, data = dat)
>coef_test(egger_multi, vcov = "CR2")
>
>Because I conducted a multivariate meta-analysis assuming rho = 0.8, I wonder for
>the Egger’s test, Do I need to let V equals to the imputed covariance matrix?
>Would anyone help me to see if my following code is correct? Thanks.
>
>V_listm <- impute_covariance_matrix(vi = meta$dv,
>                                    cluster = meta$Study.ID,
>                                    r = 0.8)
>egger_multi <- rma.mv(yi =Cohen.s.d, V = V_listm, random = ~ 1 | Study.ID/IID,
>                      mods = ~ sqrt(dv), data = meta)
>coef_test(egger_multi, vcov = "CR2")

If you used such an approximate V matrix for your analyses, then I would also use this in this model.

>Also, I have tried V = V_listm and V = dv, but it gave me different results. When
>I use V = V_Vlistm, my results suggest the effect was no longer statistically
>significant but when I use V = dv, my result is still significant.
>Does that mean my results were sensitive to the value of rho? Thanks.

Yes, although it's not clear to me what exactly you mean by "the effect". The coefficient corresponding to 'sqrt(dv)'?

>By the way, does anyone have any suggestions/codes for other methods of testing
>publication bias? Many thanks.

Just a pedantic note: There are no methods for testing for publication bias. One can for example test if there is a relationship between the observed effects and their standard errors (as done above), which could result from publication bias, but there could be other explanations for such a relationship besides publication bias.

This aside, one can also examine if there is a relationship at the study level (not at the level of the individual estimates, as done above). A simple approach for this would be to aggregate the estimates to the study level, using the aggregate() function. In fact, at that point, you could apply all of the methods available in metafor or other packages related to the issue of publication bias (including things like trim-and-fill, selection models, and so on).

Best,
Wolfgang