[R-meta] Analysis of publication bias on rma.mv object

[Viechtbauer, Wolfgang (SP)]

Dear Man Hey,

If the data have a multilevel structure, then I think one should analyze them as such, whether one is conducting the 'main' analysis or some kind of publication bias analysis. 

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of CHIU, Man Hey
Sent: Wednesday, 18 July, 2018 8:30
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] Analysis of publication bias on rma.mv object

Dear Dr. Viechtbauer and all,

Thank you Dr. Viechtbauer for answering my previous question on multilevel
meta-analysis. It is detailed and helped me a lot.

I ran into other problems when I did the publication bias analysis. As we
all know, regtest() and trimfill() have not been implemented for rma.mv
objects. But I know to implement Egger's regression test, we can simply
enter the standard error as a moderator to the rma.mv() function. For
example:

R> rma.mv(yi, vi, mods = Standard_Error, random = ~1 | Study/Outcome)

Yet, you have mentioned in another thread in 2017 that trim and fill had
not been extended to this kind of complex model and we should not do the
trim and fill on the rma.mv object. However, I still read some papers that
would like to do the trim and fill even when their primary analysis was a
multilevel model. For example, in Kredlow and colleagues' (2016) study,
they mentioned that "(For publication bias analysis,) to examine the full
pattern of effects, a multilevel approach was not used and these analyses
were conducted in Comprehensive Meta-Analysis with each effect treated as
independent. (p.318)". Precisely, their main approach was the multilevel
model, but they did the publication bias analysis by re-entering the data
into a simple model.

What do you think about this approach? Can someone run a simple random
effect model to a dataset, which originally was fit into a multilevel
model, to conduct the publication bias analysis (specifically the
regression test and trim and fill method)? Are this analysis still
meaningful to the results of the original multilevel model?

Regards,
Man Hey

References:
Kredlow, M. A., Unger, L. D., & Otto, M. W. (2016). Harnessing
reconsolidation to weaken fear and appetitive memories: A meta-analysis of
post-retrieval extinction effects. *Psychological Bulletin*, *142*(3), 314.