[R-meta] Funnel Plots for Multilevel Meta

[Dr. Gerta Rücker]

Dear Valeria,

You find discussion of these issues in the literature, for example:

https://www.bmj.com/content/343/bmj.d4002 (Sterne et al. 2011, 
Recommendations for examining and interpreting funnel plot asymmetry in 
meta-analyses) - One rule of thumb is that looking at a funnel plot 
doesn't make much sense with less than 10 effects.

https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1452 (Marks-Anglin 
& Chen 2020, A historical review of publication bias)

https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1468 (Page et al. 2020, 
Investigating and dealing with publication bias and other reporting 
biases in meta‐analyses of health research: A review)

Best,

Gerta

Am 12.12.2020 um 14:43 schrieb Valeria Ivaniushina:
> Dear Michael and Wolfgang.
>
> at the end of Egger's regression / funnel plot thread, Wolfgang said: "To
> what extent such a plot is indicative of publication bias / small study
> effects is a different issue"
>
> Could you please say more on this topic?
> Particularly I am interested in this: is a plot with 8 effects informative?
> Or a plot with less than 20 effects?
>
> Best,
> Valeria
>
> On Fri, Dec 11, 2020 at 11:28 PM Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>
>> A funnel plot is simply a plot of the estimates against their standard
>> errors (or some other measure of precision). So one can draw such a plot
>> whether there are multiple estimates from the same study or not. Hence,
>> funnel() in metafor will happily do so:
>>
>> library(metafor)
>>
>> dat <- dat.konstantopoulos2011
>> res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
>> res
>> funnel(res)
>>
>> One could indicate (with different colors or plotting symbols) which
>> estimates belong to the same study.
>>
>> cols <- palette.colors(length(unique(dat$district)), palette="Alphabet")
>> cols <- cols[as.numeric(factor(dat$district))]
>> funnel(res, col=cols)
>>
>> Then one can see how points from the same study (or in this case,
>> 'district') cluster together.
>>
>> To what extent such a plot is indicative of publication bias / small study
>> effects is a different issue (but the same applies even to simpler
>> meta-analyses with a single estimate per study).
>>
>> Best,
>> Wolfgang
>>
>>> -----Original Message-----
>>> From: R-sig-meta-analysis [mailto:
>> r-sig-meta-analysis-bounces using r-project.org]
>>> On Behalf Of Dylan Johnson
>>> Sent: Friday, 11 December, 2020 20:03
>>> To: Michael Dewey; r-sig-meta-analysis using r-project.org
>>> Subject: Re: [R-meta] Funnel Plots for Multilevel Meta
>>>
>>> Hi Michael,
>>>
>>> Would it not be nonsensical to have multiple effects from the same article
>>> in the funnel plot though?
>>>
>>> With the Egger's regression I was able to accommodate the fact that their
>> is
>>> nonindependence of the effects, but am unsure how to proceed with a funnel
>>> plot.
>>>
>>> Dylan
>>>
>>> Dylan Johnson, MSc
>>>
>>> MA Student, School and Clinical Child Psychology
>>> Department of Applied Psychology and Human Development
>>>
>>> University of Toronto
>>> 252 Bloor Street West
>>>
>>> Toronto, ON M5S 1V6
>>>
>>> From: Michael Dewey<mailto:lists using dewey.myzen.co.uk>
>>> Sent: December 11, 2020 6:12 AM
>>> To: Dylan Johnson<mailto:dylanr.johnson using mail.utoronto.ca>; r-sig-meta-
>>> analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
>>> Subject: Re: [R-meta] Funnel Plots for Multilevel Meta
>>>
>>> Dear Dylan
>>>
>>> Perhaps I misunderstand you but if you have the data for a regression
>>> type test like Egger's do you not just plot that? The funnel() function
>>> in metafor does that and I am sure equivalent solutions can be found in
>>> meta and many other packages.
>>>
>>> Michael
>>>
>>> On 11/12/2020 01:14, Dylan Johnson wrote:
>>>> Hello,
>>>>
>>>> Thanks everyone for helping me sort out the Egger's test with
>> multi-level
>>> meta modelling!
>>>> Is there any option in R for producing Funnel plots that are appropriate
>>> for the nonindependence? I imagine that the standard funnel plot would be
>>> deceiving if it came from a multi-level design.
>>>> Many thanks!
>>>>
>>>> Best,
>>>> Dylan
>>>>
>>>> Dylan Johnson, MSc
>>>>
>>>> MA Student, School and Clinical Child Psychology
>>>> Department of Applied Psychology and Human Development
>>>>
>>>> University of Toronto
>>>> 252 Bloor Street West
>>>>
>>>> Toronto, ON M5S 1V6
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis