[R-meta] Results interpretation publication bias

[Michael Dewey]

Dear Wilma

Please keep the list copied in as other may be able to help better than me.

Those funnel plots look very odd. I would normally have expected to see 
the pseudo confidence bands plotted about the estimated value, not zero. 
Apart from a few studies of low precision everything else is in an 
almost rectangular block which does not suggest to me evidence of small 
study effects.

I am attaching your plot in case someone else has ideas.

Michael

On 03/04/2023 15:47, Wilma Charlott Theilig wrote:
> Dear Michael,
> 
> those were some really helpful tips, thanks!
> 
> I have attached the funnel plot for the trim and fill method with and 
> without outlier as a PDF.
> 
> 
> Greetings Wilma
> 
> ------------------------------------------------------------------------
> *Von:* Michael Dewey <lists using dewey.myzen.co.uk>
> *Gesendet:* Montag, 3. April 2023 14:55:06
> *An:* R Special Interest Group for Meta-Analysis
> *Cc:* Theilig, Wilma Charlott
> *Betreff:* Re: [R-meta] Results interpretation publication bias
> Dear Wilma
> 
> Comments in-line
> 
> On 03/04/2023 10:14, Wilma Charlott Theilig via R-sig-meta-analysis wrote:
>> Hey all,
>> 
>> I am currently conducting my first meta-analysis using a random effects model. I wanted to investigate whether there is publication bias in my studies using different small-study effects methods. Egger's test and rank correlation test indicate funnel plot  asymmetry. I then used the trim and fill method twice.
> 
> If you are interested in publication bias, as opposed to small-study
> effects more generally, then
> https://doi.org/10.1016/j.jclinepi.2009.05.008 
> <https://doi.org/10.1016/j.jclinepi.2009.05.008>
> suggests that a selection model would be preferable to trim and fill.
> 
>> Once with all studies and since I had a very high I2 value (98.4%) and then again but with excluded outliers (here I used the find.outlier function from dmetar to identify 39 of 79 studies as outliers) as a sensitivity analysis.
> 
> A method which identifies the majority of your data as outliers is not
> detecting outliers in the normal meaning of the word. Is it telling you
> you have a mixture distribution of effect sizes? When you look at them
> do you see any common feature about the primary studies? That is where I
> would look first to try to identify possible moderators.
> 
>> Both values are higher (r= .57 and r= .38 without outliers) than than the original overall pooled effect size (r= .36). So my interpretation would be: there seems to be "missing" mainly large effect sizes with small standard errors (i.e. large studies reporting  large effect sizes, at least according to the contour enhanced funnel 
> plot).
> 
> Although this list deprecates attachments actually seeing the funnel
> plot might help here. I think the list might accept pdf attachments.
> 
> Michael
> 
>    But maybe it is also due to high heterogeneity? How can I interpret
> this result? I would greatly appreciate an answer a
>>   nd some help.
>> 
>> Best regards
>> 
>> Wilma
>> 
>>        [[alternative HTML version deleted]]
>> 
>> _______________________________________________
>> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
>> To manage your subscription to this mailing list, go to:
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis 
> <https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis>
>> 
> 
> -- 
> Michael
> http://www.dewey.myzen.co.uk/home.html 
> <http://www.dewey.myzen.co.uk/home.html>
> 
> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>	Virus-free.www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
> 
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

-- 
Michael
http://www.dewey.myzen.co.uk/home.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Funnel-plot-TF-TF-no-Outliers.pdf
Type: application/pdf
Size: 673071 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20230403/61d1c041/attachment-0001.pdf>