[R-meta] Publication bias/sensitivity analysis in multivariate meta-analysis

[Gerta Ruecker]

Dear Norman, dear all,

To clarify the notions:

Small-study effects: All effects manifesting themselves as small studies 
having different effects from large studies. The notion was coined by 
Sterne et al. (Sterne, J. A. C., Gavaghan, D., and Egger, M. (2000). 
Publication and related bias in meta-analysis: Power of statistical 
tests and prevalence in the literature.
Journal of Clinical Epidemiology, 53:1119–1129.) Small-study effects are 
seen in a funnel plot as asymmetry.

Reasons for small-study effects may be: Heterogeneity, e.g., small 
studies have selected patients (for example, worse health status); 
publication bias (see below), mathematical artifacts for binary data 
(Schwarzer, G., Antes, G., and Schumacher, M. (2002). Inflation of type 
I error rate in two statistical tests for the detection of publication 
bias in meta-analyses with binary outcomes. Statistics in Medicine, 
21:2465–2477), or coincidence.

Publication bias is one possible reason of small-study effects and means 
that small studies with small, no, or undesired effects are not 
published and therefore not found in the literature. The result is an 
effect estimate that is biased towards large effects.

Sensitivity analysis is a possibility to investigate small-study 
effects. There is an abundance of literature and methods how to do this. 
Well-known models are selection models, e.g. Vevea, J. L. and Hedges, L. 
V. (1995). A general linear model for estimating effect size in the 
presence of publication bias. Psychometrika, 60:419–435 or Copas, J. and 
Shi, J. Q. (2000). Meta-analysis, funnel plots and sensitivity analysis. 
Biostatistics, 1:247–262.

I attach a talk with more details.

Best,

Gerta


Am 15.06.2020 um 02:28 schrieb Norman DAURELLE:
> Hi all, I read this thread, and the topic interests me, but I didn't quite understand your answer :when you say " Publication bias is a subset of small study effects where you know the
> aetiology of the small study effects. If you do not then it is safer to
> refer to small study effects. "
> I don't really understand what you mean.I thought publication bias meant that the studies included in a sample of study didn't really account for the whole range of possible effect sizes (with their associated standard error).Is that not what publication bias refers to ? And if it is, how does it also correspond to the definition you gave ?Thank you !Norman.
> ----- Mail d'origine -----
> De: Michael Dewey <lists using dewey.myzen.co.uk>
> À: Huang Wu <huang.wu using wmich.edu>, r-sig-meta-analysis using r-project.org
> Envoyé: Sun, 14 Jun 2020 12:54:30 +0200 (CEST)
> Objet: Re: [R-meta] Publication bias/sensitivity analysis in multivariate meta-analysis
>
> Dear Huang
>
> Comments in-line
>
> On 13/06/2020 20:57, Huang Wu wrote:
>> Hi all,
>>
>> Greetings. I have some questions about publication bias/sensitivity analysis. First, are publication bias and sensivity analysis the same thing? If not, how are they different?
> Publication bias is a subset of small study effects where you know the
> aetiology of the small study effects. If you do not then it is safer to
> refer to small study effects. A sensitivity analysis could be almost
> anything but usually it manes fitting the model to one or more data-sets
> similar to the original one. Examples are leave-one-out analysis, or
> using only a subset of supposed higher quality studies.
>
>> Second, I saw people use funnel plot, fail-safe N, Egger�s regression test to test publication bias (http://www.metafor-project.org/doku.php/features), are these methods applicable to multivariate meta-analysis?
> Yes they are.
>
> Thanks.
>> Third, what do you recommend to do publication bias/sensivity analysis in multivariate meta-analysis? Thanks
>>
> I think what analysis you do will depend on the scientific question.
>
> Michael
>
>> Best wishes
>> Huang
>>
>> Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
>>
>>
>> [[alternative HTML version deleted]]
>>
>>
>>
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
-- 

Dr. rer. nat. Gerta Rücker, Dipl.-Math.

Institute of Medical Biometry and Statistics,
Faculty of Medicine and Medical Center - University of Freiburg

Stefan-Meier-Str. 26, D-79104 Freiburg, Germany

Phone:    +49/761/203-6673
Fax:      +49/761/203-6680
Mail:     ruecker using imbi.uni-freiburg.de
Homepage: https://www.uniklinik-freiburg.de/imbi.html


-------------- next part --------------
A non-text attachment was scrubbed...
Name: SmallStudyEffects2020.pdf
Type: application/pdf
Size: 256567 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20200615/eaf13bc4/attachment-0001.pdf>