[R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and Outcomes

Michael Dewey ||@t@ @end|ng |rom dewey@myzen@co@uk
Fri Mar 24 11:48:38 CET 2023 

Dear Ruth

I think in general when you perform two analyses and they give different 
but similar results you should (a) present both, (b) present a plausible 
explanation of why. Wolfgang has given you a lead there. What ever you 
do the reader will make up their own mind which to believe but at least 
you have put all the evidence in front of them.

Michael

On 23/03/2023 19:58, Ruth Elisabeth Appel via R-sig-meta-analysis wrote:
> Hi Wolfgang,
> 
> Thank you so much for these additional notes!
> 
> On 1), yes, I used the common effects model for the pairwise MAs, so the tau^2 are all 0 and therefore the same.
> I think I also used the common effects model in the NMA object, at least I specified "fixed = TRUE". I'm providing an excerpt from my code below, do you think this is an apples-to-apples comparison then?
> My apologies if I misspecified or misinterpreted something, I went through some tutorials, but am still new to meta-analyses.
> 
> # NMA
> net1 <- netmeta(...,
>                  all.treatments = TRUE,
>                  reference.group = "Control",
>                  fixed = TRUE,
>                  random = FALSE,
>                  sm = "SMD",
>                  method.sm = "Cohen",
>                  studies = c("Study 1", "Study 2", "Study 3"),
>                  n.arms = 3,
>                  ...)
> # pairwise MA
> netpair1 <- netpairwise(net1, sep.trts = " vs. ")
> 
> # combine NMA objects to show all outcomes in one plot
> nets <- netbind(net1, net2, ..., fixed = TRUE)
> 
> 
> As for plots, I have 1 plot with NMA results for all outcomes, and 1 plot for pairwise MA for each outcome:
> 
> # NMA for all outcomes
> forest(nets,
>         text.fixed = "Common Fixed Effects Model",
>         ...)
> 
> # pairwise MA for one outcome
> netpair1  %>%
>    forest(text.fixed = "Common Fixed Effects Model", ...)
> 
> On 2), yes, I think what I meant is that I observe a significant effect with the NMA, but not the pairwise MA, so I meant to say this effect is not robust to different MA specifications. The wording you suggested is great, so maybe I could say the effect is significant when using NMA, which provides stronger evidence – assuming the assumptions for NMA are met – because it can take indirect evidence into account as well, but the effect is not significant in a pairwise MA that only relies on direct evidence.
> 
> Best,
> Ruth
> 
> Ruth Elisabeth Appel
> Ph.D. Candidate in Political Communication and Media Psychology
> Stanford University
> rappel using stanford.edu<mailto:rappel using stanford.edu>
> 
> On Mar 23, 2023, at 1:07 AM, Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>> wrote:
> 
> Hi all,
> 
> Just two notes:
> 
> 1) If you used the common-effects model for the pairwise MAs, then tau^2 is 0 for these models by definition (so it sounds a bit odd to say that "the tau^2 should be the same"). But I assume you did not use a common-effects model for the NMA, so now we are comparing apples and oranges.
> 
> 2) I don't think it is quite right to say that the significance of the effect in the pairwise MA is not "robust". It is perfectly fine, being based on the direct evidence. In fact, one could argue that the NMA is potentially less robust, since it makes assumptions about the consistency of the evidence. A better way to state this is to say that, if the assumptions underling the NMA are correct (and taking into consideration what Gerta wrote about tau^2), then the NMA can provide stronger evidence, because it can also take the indirect evidence into consideration.
> 
> Best,
> Wolfgang
> 
> -----Original Message-----
> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
> Behalf Of Ruth Elisabeth Appel via R-sig-meta-analysis
> Sent: Thursday, 23 March, 2023 6:52
> To: Dr. Gerta Rücker
> Cc: Ruth Elisabeth Appel; r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and
> Outcomes
> 
> Hi Gerta,
> 
> Thank you so much, this is incredibly helpful!
> 
> I used the common effects model for the pairwise MAs, so the tau^2 should be the
> same, which implies that in this case the NMA estimates should be the ones with
> higher precision. I think I will explain this logic in the paper then, i.e. that
> the NMA estimate should be more precise, but the significance of the effect is
> not robust in the case of a pairwise MA that only takes direct evidence into
> account.
> 
> Thank you for pointing out the use of forest(netsplit(net1)), this is great to
> see the different estimates in the same plot!
> 
> Best,
> Ruth
> 
> Ruth Elisabeth Appel
> Ph.D. Candidate in Political Communication and Media Psychology
> Stanford University
> rappel using stanford.edu<mailto:rappel using stanford.edu><mailto:rappel using stanford.edu>
> 
> On Mar 22, 2023, at 3:59 AM, Dr. Gerta Rücker <gerta.ruecker using uniklinik-
> freiburg.de<http://freiburg.de><mailto:gerta.ruecker using uniklinik-freiburg.de>> wrote:
> 
> Dear Ruth,
> 
> Here are my answers, To your first question:
> 
> If an effect was significant with the NMA estimates (TE.nma.fixed from netmeta
> object), but not the direct fixed effect (TE.direct.fixed from netmeta object) in
> a pairwise comparison, would this suggest that the interpretation should be that
> there is a significant effect, but it is not robust? In terms of substantive
> interpretation, would this mean there is insufficient direct evidence? And is it
> generally true that pairwise comparisons are more conservative?
> 
> While pairwise MA uses only information from the direct comparisons, NMA
> potentially uses information from all comparisons in the network. Therefore NMA
> usually provides higher precision than MA (depending on the network structure and
> the estimate of the random effects variance). This means that NMA-based estimates
> may be „significant“ when pairwise estimates are not. This is just what one would
> expect, and it is one of the aims of NMA to obtain more precises estimates.
> 
> Note that if each pairwise MA uses its own heterogeneity estimate (tau²), it can
> be the other way round, because the tau² of the NMA may be larger than that of a
> particular pairwise estimate (for example, if there was only one study for this
> comparison). If, however, all pairwise comparisons use the same tau² (for example
> because the common effect model is used), the NMA estimate is at least as precise
> as the pairwise estimate.
> 
> Second question:
> 
> Further, if I want to use forest() to plot TE.direct.fixed instead of
> TE.nma.fixed of several netmeta objects combined with netbind(), what is the
> correct way of specifying this (i.e., in which function and with which argument)?
> (I realize the package was just updated, but I have been using the previous
> version and couldn't find the right settings yet.)
> 
> An elegant way to compare direct with indirect and network estimates in the same
> forest plot is to use netsplit(). If your NMA object is called net1, the simplest
> call is
> 
> forest(netsplit(net1))
> 
> It provides all three types of estimates in one plot, where the direct and
> indirect estimates are based on the same tau² which is estimated from the NMA. If
> you use the common effects model, tau² is set to 0 anyway. You may use the
> argument show (to choose the comparisons to be shown) and the arguments overall,
> direct and indirect (to choose the estimates to be shown). For example, with
> 
> forest(netsplit(net1), show = "with.direct", indirect = FALSE)
> 
> the output is restricted to those comparisons for which a direct estimate is
> available (argument show), each with its direct and network estimate (argument
> indirect).
> 
> Note again that in case of the random effects model the results of the pairwise
> meta-analyses depend on whether you use different estimates of tau² for each
> independent pairwise MA or take the common tau² from the NMA.
> 
> Purely separate pairwise MAs are obtained using netpairwise(), with corresponding
> forest function forest.netpairwise(), for example
> 
> forest(netpairwise(net1))
> 
> Best,
> Gerta
> 
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 
> 

-- 
Michael
http://www.dewey.myzen.co.uk/home.html

More information about the R-sig-meta-analysis mailing list