[R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and Outcomes

[Ruth Elisabeth Appel]

Hi Wolfgang,

Thank you so much for these additional notes!

On 1), yes, I used the common effects model for the pairwise MAs, so the tau^2 are all 0 and therefore the same.
I think I also used the common effects model in the NMA object, at least I specified "fixed = TRUE". I'm providing an excerpt from my code below, do you think this is an apples-to-apples comparison then?
My apologies if I misspecified or misinterpreted something, I went through some tutorials, but am still new to meta-analyses.

# NMA
net1 <- netmeta(...,
                all.treatments = TRUE,
                reference.group = "Control",
                fixed = TRUE,
                random = FALSE,
                sm = "SMD",
                method.sm = "Cohen",
                studies = c("Study 1", "Study 2", "Study 3"),
                n.arms = 3,
                ...)
# pairwise MA
netpair1 <- netpairwise(net1, sep.trts = " vs. ")

# combine NMA objects to show all outcomes in one plot
nets <- netbind(net1, net2, ..., fixed = TRUE)


As for plots, I have 1 plot with NMA results for all outcomes, and 1 plot for pairwise MA for each outcome:

# NMA for all outcomes
forest(nets,
       text.fixed = "Common Fixed Effects Model",
       ...)

# pairwise MA for one outcome
netpair1  %>%
  forest(text.fixed = "Common Fixed Effects Model", ...)

On 2), yes, I think what I meant is that I observe a significant effect with the NMA, but not the pairwise MA, so I meant to say this effect is not robust to different MA specifications. The wording you suggested is great, so maybe I could say the effect is significant when using NMA, which provides stronger evidence – assuming the assumptions for NMA are met – because it can take indirect evidence into account as well, but the effect is not significant in a pairwise MA that only relies on direct evidence.

Best,
Ruth

Ruth Elisabeth Appel
Ph.D. Candidate in Political Communication and Media Psychology
Stanford University
rappel using stanford.edu<mailto:rappel using stanford.edu>

On Mar 23, 2023, at 1:07 AM, Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>> wrote:

Hi all,

Just two notes:

1) If you used the common-effects model for the pairwise MAs, then tau^2 is 0 for these models by definition (so it sounds a bit odd to say that "the tau^2 should be the same"). But I assume you did not use a common-effects model for the NMA, so now we are comparing apples and oranges.

2) I don't think it is quite right to say that the significance of the effect in the pairwise MA is not "robust". It is perfectly fine, being based on the direct evidence. In fact, one could argue that the NMA is potentially less robust, since it makes assumptions about the consistency of the evidence. A better way to state this is to say that, if the assumptions underling the NMA are correct (and taking into consideration what Gerta wrote about tau^2), then the NMA can provide stronger evidence, because it can also take the indirect evidence into consideration.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
Behalf Of Ruth Elisabeth Appel via R-sig-meta-analysis
Sent: Thursday, 23 March, 2023 6:52
To: Dr. Gerta Rücker
Cc: Ruth Elisabeth Appel; r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
Subject: Re: [R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and
Outcomes

Hi Gerta,

Thank you so much, this is incredibly helpful!

I used the common effects model for the pairwise MAs, so the tau^2 should be the
same, which implies that in this case the NMA estimates should be the ones with
higher precision. I think I will explain this logic in the paper then, i.e. that
the NMA estimate should be more precise, but the significance of the effect is
not robust in the case of a pairwise MA that only takes direct evidence into
account.

Thank you for pointing out the use of forest(netsplit(net1)), this is great to
see the different estimates in the same plot!

Best,
Ruth

Ruth Elisabeth Appel
Ph.D. Candidate in Political Communication and Media Psychology
Stanford University
rappel using stanford.edu<mailto:rappel using stanford.edu><mailto:rappel using stanford.edu>

On Mar 22, 2023, at 3:59 AM, Dr. Gerta Rücker <gerta.ruecker using uniklinik-
freiburg.de<http://freiburg.de><mailto:gerta.ruecker using uniklinik-freiburg.de>> wrote:

Dear Ruth,

Here are my answers, To your first question:

If an effect was significant with the NMA estimates (TE.nma.fixed from netmeta
object), but not the direct fixed effect (TE.direct.fixed from netmeta object) in
a pairwise comparison, would this suggest that the interpretation should be that
there is a significant effect, but it is not robust? In terms of substantive
interpretation, would this mean there is insufficient direct evidence? And is it
generally true that pairwise comparisons are more conservative?

While pairwise MA uses only information from the direct comparisons, NMA
potentially uses information from all comparisons in the network. Therefore NMA
usually provides higher precision than MA (depending on the network structure and
the estimate of the random effects variance). This means that NMA-based estimates
may be „significant“ when pairwise estimates are not. This is just what one would
expect, and it is one of the aims of NMA to obtain more precises estimates.

Note that if each pairwise MA uses its own heterogeneity estimate (tau²), it can
be the other way round, because the tau² of the NMA may be larger than that of a
particular pairwise estimate (for example, if there was only one study for this
comparison). If, however, all pairwise comparisons use the same tau² (for example
because the common effect model is used), the NMA estimate is at least as precise
as the pairwise estimate.

Second question:

Further, if I want to use forest() to plot TE.direct.fixed instead of
TE.nma.fixed of several netmeta objects combined with netbind(), what is the
correct way of specifying this (i.e., in which function and with which argument)?
(I realize the package was just updated, but I have been using the previous
version and couldn't find the right settings yet.)

An elegant way to compare direct with indirect and network estimates in the same
forest plot is to use netsplit(). If your NMA object is called net1, the simplest
call is

forest(netsplit(net1))

It provides all three types of estimates in one plot, where the direct and
indirect estimates are based on the same tau² which is estimated from the NMA. If
you use the common effects model, tau² is set to 0 anyway. You may use the
argument show (to choose the comparisons to be shown) and the arguments overall,
direct and indirect (to choose the estimates to be shown). For example, with

forest(netsplit(net1), show = "with.direct", indirect = FALSE)

the output is restricted to those comparisons for which a direct estimate is
available (argument show), each with its direct and network estimate (argument
indirect).

Note again that in case of the random effects model the results of the pairwise
meta-analyses depend on whether you use different estimates of tau² for each
independent pairwise MA or take the common tau² from the NMA.

Purely separate pairwise MAs are obtained using netpairwise(), with corresponding
forest function forest.netpairwise(), for example

forest(netpairwise(net1))

Best,
Gerta


	[[alternative HTML version deleted]]