[R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and Outcomes

[Viechtbauer, Wolfgang (NP)]

Hi all,

Just two notes:

1) If you used the common-effects model for the pairwise MAs, then tau^2 is 0 for these models by definition (so it sounds a bit odd to say that "the tau^2 should be the same"). But I assume you did not use a common-effects model for the NMA, so now we are comparing apples and oranges.

2) I don't think it is quite right to say that the significance of the effect in the pairwise MA is not "robust". It is perfectly fine, being based on the direct evidence. In fact, one could argue that the NMA is potentially less robust, since it makes assumptions about the consistency of the evidence. A better way to state this is to say that, if the assumptions underling the NMA are correct (and taking into consideration what Gerta wrote about tau^2), then the NMA can provide stronger evidence, because it can also take the indirect evidence into consideration.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Ruth Elisabeth Appel via R-sig-meta-analysis
>Sent: Thursday, 23 March, 2023 6:52
>To: Dr. Gerta Rücker
>Cc: Ruth Elisabeth Appel; r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and
>Outcomes
>
>Hi Gerta,
>
>Thank you so much, this is incredibly helpful!
>
>I used the common effects model for the pairwise MAs, so the tau^2 should be the
>same, which implies that in this case the NMA estimates should be the ones with
>higher precision. I think I will explain this logic in the paper then, i.e. that
>the NMA estimate should be more precise, but the significance of the effect is
>not robust in the case of a pairwise MA that only takes direct evidence into
>account.
>
>Thank you for pointing out the use of forest(netsplit(net1)), this is great to
>see the different estimates in the same plot!
>
>Best,
>Ruth
>
>Ruth Elisabeth Appel
>Ph.D. Candidate in Political Communication and Media Psychology
>Stanford University
>rappel using stanford.edu<mailto:rappel using stanford.edu>
>
>On Mar 22, 2023, at 3:59 AM, Dr. Gerta Rücker <gerta.ruecker using uniklinik-
>freiburg.de<mailto:gerta.ruecker using uniklinik-freiburg.de>> wrote:
>
>Dear Ruth,
>
>Here are my answers, To your first question:
>
>>  If an effect was significant with the NMA estimates (TE.nma.fixed from netmeta
>object), but not the direct fixed effect (TE.direct.fixed from netmeta object) in
>a pairwise comparison, would this suggest that the interpretation should be that
>there is a significant effect, but it is not robust? In terms of substantive
>interpretation, would this mean there is insufficient direct evidence? And is it
>generally true that pairwise comparisons are more conservative?
>
>While pairwise MA uses only information from the direct comparisons, NMA
>potentially uses information from all comparisons in the network. Therefore NMA
>usually provides higher precision than MA (depending on the network structure and
>the estimate of the random effects variance). This means that NMA-based estimates
>may be „significant“ when pairwise estimates are not. This is just what one would
>expect, and it is one of the aims of NMA to obtain more precises estimates.
>
>Note that if each pairwise MA uses its own heterogeneity estimate (tau²), it can
>be the other way round, because the tau² of the NMA may be larger than that of a
>particular pairwise estimate (for example, if there was only one study for this
>comparison). If, however, all pairwise comparisons use the same tau² (for example
>because the common effect model is used), the NMA estimate is at least as precise
>as the pairwise estimate.
>
>Second question:
>
>>  Further, if I want to use forest() to plot TE.direct.fixed instead of
>TE.nma.fixed of several netmeta objects combined with netbind(), what is the
>correct way of specifying this (i.e., in which function and with which argument)?
>(I realize the package was just updated, but I have been using the previous
>version and couldn't find the right settings yet.)
>
>An elegant way to compare direct with indirect and network estimates in the same
>forest plot is to use netsplit(). If your NMA object is called net1, the simplest
>call is
>
>forest(netsplit(net1))
>
>It provides all three types of estimates in one plot, where the direct and
>indirect estimates are based on the same tau² which is estimated from the NMA. If
>you use the common effects model, tau² is set to 0 anyway. You may use the
>argument show (to choose the comparisons to be shown) and the arguments overall,
>direct and indirect (to choose the estimates to be shown). For example, with
>
>forest(netsplit(net1), show = "with.direct", indirect = FALSE)
>
>the output is restricted to those comparisons for which a direct estimate is
>available (argument show), each with its direct and network estimate (argument
>indirect).
>
>Note again that in case of the random effects model the results of the pairwise
>meta-analyses depend on whether you use different estimates of tau² for each
>independent pairwise MA or take the common tau² from the NMA.
>
>Purely separate pairwise MAs are obtained using netpairwise(), with corresponding
>forest function forest.netpairwise(), for example
>
>forest(netpairwise(net1))
>
>Best,
>Gerta