[R-meta] Meta-Analysis and Forest Plot for Multiple Treatments and Outcomes

[Dr. Gerta Rücker]

Hi Ruth,

There is another function in netmeta you may want to use to have all 
your 8 outcomes in one forest plot: function netbind() which is to 
bundle the results of several network meta-analyses into one forest 
plot. Here I would take the NMA estimates, not the pairwise direct 
comparisons.

I cannot really answer your question related to the correlation between 
outcomes. This is because I am working in the medical field, also 
Cochrane, where it is quite unusual to put all outcomes into one model, 
because we almost never have any knowledge about the within-study 
correlations - thus the outcomes are usually analyzed separately (they 
also are on different scales, we rarely use SMD). A paper discussing 
multivariate meta-analysis is 
https://onlinelibrary.wiley.com/doi/10.1002/sim.4172 (with discussion).

Best,

Gerta



Am 01.02.2022 um 06:40 schrieb Ruth Appel:
> Hi Gerta,
>
> Thank you so much for your super helpful and quick reply!
> Yes, that is correct, I used the netmeta package as well (I considered 
> it a complement/extension of meta [part of the yet to be established 
> metaverse ;)], but I should have mentioned all packages I was using). 
> The combination of netpairwise() and forest() is very close to what I 
> was looking for – it would only be perfect if I could plot all 8 
> outcomes in the same plot rather than showing 8 separate plots, and I 
> am not sure whether that’s possible since netpairwise seems to 
> configure the different comparisons as subgroups and I couldn’t see 
> another option to specify that I would like to show effects for 
> several outcomes.
>
> That is an important note regarding potential inconsistency issues 
> with Hedges’ g, I could use Cohen’s d in that case.
>
> Regarding the correlation between outcomes, how strong could it 
> potentially bias the results in your experience? I think the 
> netpairwise() solution is great, so if the bias introduced is not too 
> big, I might use that approach.
>
> Best,
> Ruth
>
> *Ruth Elisabeth Appel*
> Ph.D. Candidate in Media Psychology
> Stanford University Department of Communication
> rappel using stanford.edu <mailto:rappel using stanford.edu>
>
>> On Jan 31, 2022, at 10:34 AM, Dr. Gerta Rücker 
>> <ruecker using imbi.uni-freiburg.de <mailto:ruecker using imbi.uni-freiburg.de>> 
>> wrote:
>>
>> Hi Ruth,
>>
>> First of all, if I understand it correctly, what you are aiming at is 
>> a network meta-analysis (NMA). Therefore, meta is not the appropriate 
>> R package, which would be netmeta (specialized to NMA) or metafor 
>> (more general). It seems you have in fact used netmeta, because you 
>> write about a netmeta object, is that true? I would see the NMA as 
>> the primary analysis and the pairwise meta-analyses as sensitivity 
>> analyses. These can be conducted using function netpairwise() in 
>> netmeta; for the fixed effect model, also netsplit() should provide 
>> the direct pairwise comparisons. Perhaps @Guido Schwarzer sees a 
>> convenient way to visualize the results within the same forest plot 
>> using forest.netsplit().
>>
>> I would expect a problem with Hedges' g for three-arm studies because 
>> the results within a trial may become inconsistent (this holds for 
>> Hedges' g, but not for Cohen's d, as implemented in netmeta).
>>
>> Note that netmeta accounts for multiple comparisons between groups 
>> with a study, however, it does not handle multivariate outcomes. 
>> Thus, if you want to account for correlation between outcomes, you 
>> need metafor. With respect to metafor, others are more expert than me.
>>
>> Best,
>>
>> Gerta
>>
>> Am 31.01.2022 um 19:02 schrieb Ruth Appel:
>>> Hi all,
>>>
>>> I’m currently conducting my first meta-analysis, an internal 
>>> meta-analysis to summarize the result of 3 similar studies my 
>>> colleagues and I conducted.
>>>
>>> I looked at the documentation of various meta-analysis packages and 
>>> tutorials, but I am still not fully sure about the best approach.
>>> The experiments I’m analyzing all have a similar structure (2 
>>> treatment groups, 1 control group; 8 different outcomes (measuring 
>>> different constructs)). The raw data has repeated measures, but we 
>>> look at outcomes at the group level, so I calculated all necessary 
>>> summary statistics (mean, sd, n).
>>> My goal is to create a forest plot that shows Hedges’ g estimated 
>>> using an FE model (because the studies were highly similar) for (1) 
>>> all 3 studies individually and (2) across all studies. Ideally, the 
>>> final result would be a single forest plot with individual study 
>>> estimates and across study estimate grouped by outcome.
>>>
>>> I managed to create such a plot with the meta package for the 2 
>>> treatment groups separately, but I realized that my SEs could be 
>>> biased in this case because I’m not accounting for the correlations 
>>> in the variance resulting from the comparison of two treatment 
>>> groups to the same control group. Similarly, I found a workaround to 
>>> show all outcomes in 1 forest plot by using subgroups for the 
>>> different outcomes, but I do not take into consideration that 
>>> outcomes might be correlated within studies. I also didn’t find a 
>>> way to show the individual study results in addition to the overall 
>>> network results in a forest plot of a netmeta object.
>>>
>>> I then tried to calculate the correct values using metafor and 
>>> following the tutorial at 
>>> https://www.metafor-project.org/doku.php/analyses:gleser2009#multiple-treatment_studies 
>>> <https://www.metafor-project.org/doku.php/analyses:gleser2009#multiple-treatment_studies>, 
>>> but it seems like the individual studies are not correctly 
>>> identified in the output (the ids are all unique instead of matching 
>>> the study variable I had created).
>>>
>>> My questions are: (1) Did I overlook guidance somewhere on how to 
>>> exactly specify a model like the one above using the metafor, meta 
>>> (or another R) package, and generate a forest plot for it?
>>> (2) If this is not easily possible, do you think the bias introduced 
>>> should be sufficiently small such that acknowledging it and 
>>> presenting separate meta-analyses for each treatment, and a network 
>>> meta analysis with the overall effects of each treatment (separately 
>>> for each outcome) in the appendix, is acceptable? (I had very 
>>> similar estimates across all the approaches described above.)
>>>
>>> Best regards, and thank you very much for your guidance,
>>> Ruth
>>>
>>> Ruth Elisabeth Appel
>>> Ph.D. Candidate in Media Psychology
>>> Stanford University Department of Communication
>>> rappel using stanford.edu <mailto:rappel using stanford.edu>
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>> -- 
>>
>> Dr. rer. nat. Gerta Rücker, Dipl.-Math.
>>
>> Guest Scientist
>> Institute of Medical Biometry and Statistics,
>> Faculty of Medicine and Medical Center - University of Freiburg
>>
>> Zinkmattenstr. 6a, D-79108 Freiburg, Germany
>>
>> Mail: ruecker using imbi.uni-freiburg.de <mailto:ruecker using imbi.uni-freiburg.de>
>> Homepage: 
>> https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker 
>> <https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker>
>>
>
-- 

Dr. rer. nat. Gerta Rücker, Dipl.-Math.

Guest Scientist
Institute of Medical Biometry and Statistics,
Faculty of Medicine and Medical Center - University of Freiburg

Zinkmattenstr. 6a, D-79108 Freiburg, Germany

Mail:     ruecker using imbi.uni-freiburg.de
Homepage: https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker


	[[alternative HTML version deleted]]