[R-meta] Publication bias in the context of subgroup analysis

[Viechtbauer, Wolfgang (NP)]

Hi Daniel,

I am not sure why you posted the same question twice -- I'll respond to this one.

In what sense do you mean 'subgroup analysis'? To me, a subgroup analysis is one where studies are split into subgroups and one fits models within these subgroups. This is not what you are doing below, at least the code doesn't reflect this.

If your question is about using something like SE (or SE^2) as a predictor in the context of a model that includes other moderators, then the answer is yes. A relevant article in this regard is:

Coburn, K. M., & Vevea, J. L. (2015). Publication bias as a function of study characteristics. Psychological Methods, 20(3), 310-330. https://doi.org/10.1037/met0000046

It doesn't discuss multilevel models, but the point is that sometimes there might be a relationship between the effect sizes and the standard errors because of some confounding with other variables. For example, suppose we have two sets of studies in our meta-analysis, the first consisting of RCTs, the other non-randomized studies. Furthermore, assume that there is no publication bias going on. But now suppose that RCTs (which tend to be smaller due to the additional complexities of running a trial) tend to yield smaller (or larger) effects than the non-randomized studies (maybe the non-randomized studies are yielding somewhat biased estimates). In this situation, the funnel plot may look asymmetric, not because of publication bias, but because of the confounding of the SEs with study type. When including study type as a moderators, then this confounding is controlled for and we find no association between the effect sizes and SEs (rightly so). This shows that it may actually be very important to account for relevant moderators before examining the data for a relationship between the SEs and the effect sizes.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Daniel Foster via R-sig-meta-analysis
> Sent: Sunday, December 10, 2023 18:48
> To: Daniel Foster via R-sig-meta-analysis <r-sig-meta-analysis using r-project.org>
> Cc: Daniel Foster <daniel.foster using utoronto.ca>
> Subject: [R-meta] Publication bias in the context of subgroup analysis
>
> Hello all,
>
> I am carrying out a multi-level meta-analysis and I have a question regarding
> publication bias in the context of subgroup analysis.
>
> I have noticed that many meta-regression models include a precision measure (SE
> or SE^2) in their final model along with covariates. Is the same true for
> subgroup analysis in a multi-level analysis wherein moderators are tested
> individually?
>
> For example:
> moderertorX <- rma.mv(yi, vi, mods = ~ SE + moderertorX, random = list(~ 1 |
> EffectSize_ID, ~1 | Study_ID), tdist=TRUE, data=data)
>
> Thanks!
> Daniel