[R-meta] Overlapping CIs with significant difference among subgroups

Gerta Ruecker ruecker @end|ng |rom |mb|@un|-|re|burg@de
Thu Jun 4 11:31:51 CEST 2020

Dear Rafael,

First of all, the information content of standard errors and confidence 
intervals is identical, they can be transformed into each other. 
Secondly, to present standard errors in a graph, one would probably show 
x ± SE(x) instead of x ± 1.96*SE(x). But what would be the advantage? 
The interpretation of this intercval would mean that the true value is 
covered by 68% of all such intervals (=1-2*(1-pnorm(1))). I don't think 
that this is of more interest than a confidence interval.

The main aim of a forest plot is interval estimation, not statistically 
comparing different studies.



Am 04.06.2020 um 08:26 schrieb Rafael Rios:
> Dear Dr. Wolfgang,
> Thank you for the feedback. I was wondering why meta-analysts did not
> exhibit standard errors instead of confidence intervals in graphs. I can
> understand the importance of showing that CIs did not include zero, but
> standard errors can be more informative when comparing subgroups of a
> moderator. This is just a curiosity.
> Best wishes,
> Rafael.
> Em qua, 3 de jun de 2020 às 05:02, Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> escreveu:
>> Dear Rafael,
>> What specifically do you mean by "this practice"? Presenting estimated
>> (average) effects with their CIs when subgrouping the studies based on some
>> categorical variable? Indeed, one cannot directly infer based on the CIs
>> whether the subgroups are actually different from each other. For this, one
>> should conduct a proper test of subgroup differences. One can also directly
>> test whether the difference between two effects is significant or not or
>> present an estimate of the difference between two effects with a
>> corresponding CI (and if that CI excludes 0, then one knows that the test
>> of the difference is significant at alpha = (100 - CI level)/100). But I
>> see nothing generally wrong with the practice of presenting subgroup
>> effects with CIs.
>> Best,
>> Wolfgang
>>> -----Original Message-----
>>> From: Rafael Rios [mailto:biorafaelrm using gmail.com]
>>> Sent: Wednesday, 03 June, 2020 5:27
>>> To: Viechtbauer, Wolfgang (SP)
>>> Cc: r-sig-meta-analysis using r-project.org
>>> Subject: Re: Overlapping CIs with significant difference among subgroups
>>> Dear Dr. Wolfgang,
>>> Thank you very much! Since confidence intervals are not very informative
>> to
>>> exhibit diferences between subgroups, why is this practice so common among
>>> meta-analysts? Why not to present standard errors instead of CIs?
>>> Best wishes,
>>> Rafael.
>>> Em ter, 2 de jun de 2020 às 03:48, Viechtbauer, Wolfgang (SP)
>>> <wolfgang.viechtbauer using maastrichtuniversity.nl> escreveu:
>>> Dear Rafael,
>>> CIs can overlap and yet the difference between the two levels can be
>>> significant. See, for example:
>>> https://towardsdatascience.com/why-overlapping-confidence-intervals-mean-
>>> nothing-about-statistical-significance-48360559900a?gi=b673a691634d
>>> https://www.psychologicalscience.org/observer/understanding-confidence-
>>> intervals-cis-and-effect-size-estimation
>>> https://blog.minitab.com/blog/real-world-quality-improvement/common-
>>> statistical-mistakes-you-should-avoid
>>> and many more (just google for "test difference overlapping confidence
>>> intervals" or something along those lines). They don't talk about meta-
>>> analysis per se, but it's the same principle.
>>> So, you can trust the test of the difference between the levels of the
>>> moderators.
>>> Best,
>>> Wolfgang
>>>> -----Original Message-----
>>>> From: Rafael Rios [mailto:biorafaelrm using gmail.com]
>>>> Sent: Monday, 01 June, 2020 21:54
>>>> To: r-sig-meta-analysis using r-project.org; Viechtbauer, Wolfgang (SP)
>>>> Subject: Overlapping CIs with significant difference among subgroups
>>>> ATTACHMENT(S) REMOVED: dataset.csv | pruned_super-tree.tre | script.R
>>>> Dear Wolfgang and All,
>>>> I conducted a multilevel mixed-effects meta-analysis and found
>> differences
>>>> between levels of two moderators. I was expecting to find non-overlapped
>>>> confidence intervals. However, I obtained overlapped confidence intervals
>>>> for all subgroups. How can I interpret these results? In such situation,
>>>> should I trust in the Q-test or in the CIs? I controlled for phylogenetic
>>>> non-independence. Is there a chance of this approach affect the
>> estimation
>>>> of CIs using predict function? My dataset and script are attached.
>>>> Best wishes,
>>>> _______________________________________________________
>>>> Prof. Dr. Rafael Rios Moura
>>>> Coordenador de Pesquisa e do NEPEE/CNPq
>>>> Laboratório de Ecologia e Zoologia (LEZ)
>>>> UEMG - Unidade Ituiutaba
>>>> ORCID: http://orcid.org/0000-0002-7911-4734
>>>> Currículo Lattes: http://lattes.cnpq.br/4264357546465157
>>>> Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
>>>> Rios de Ciência:
>> https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg
>>> --
>>> _______________________________________________________


Dr. rer. nat. Gerta Rücker, Dipl.-Math.

Institute of Medical Biometry and Statistics,
Faculty of Medicine and Medical Center - University of Freiburg

Stefan-Meier-Str. 26, D-79104 Freiburg, Germany

Phone:    +49/761/203-6673
Fax:      +49/761/203-6680
Mail:     ruecker using imbi.uni-freiburg.de
Homepage: https://www.uniklinik-freiburg.de/imbi.html

More information about the R-sig-meta-analysis mailing list