<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>To answer your question, Nelson: <br>
</p>
<p>If I have only two studies and the confidence intervals don't
overlap, I would usually present a forest plot without a pooled
estimate and discuss this in the text as indication of large
heterogeneity.</p>
<p>However, this also depends on the relevance of the difference on
the outcome scale, depending on subject-matter considerations. For
example, if I am estimating incidence rate ratios or something
similar based on very big populations, the CIs may be very short
and thus non-overlapping, but this may not be important with
respect to heterogeneity. For an example, see Figure 2b in the
attached paper (antibiotics density): The first two CIs are not
overlapping, but this doesn't seems to be a big difference. It's
only due to the enormous size of the studies.</p>
<p>Best,</p>
<p>Gerta<br>
</p>
<div class="moz-cite-prefix">Am 27.08.2020 um 18:49 schrieb Nelson
Ndegwa:<br>
</div>
<blockquote type="cite"
cite="mid:CALiySoB17ON3QeZK5QxC922Nsf9TusaKODAfH5u2bCtwe3D5jg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div>Haha, sorry, I was editing a response that included your
signature and forgot to exclude your signature :-)<br>
</div>
<div><br>
</div>
<div>nelson<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, 27 Aug 2020 at 18:47,
ne gic <<a href="mailto:negic4@gmail.com"
moz-do-not-send="true">negic4@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Wait, are you also Nelly @Nelson?</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug 27, 2020 at
6:44 PM Nelson Ndegwa <<a
href="mailto:nelson.ndegwa@gmail.com" target="_blank"
moz-do-not-send="true">nelson.ndegwa@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>Dear Gerta,</div>
<div><br>
</div>
<div>I agree with you. In the interest of playing the
devil's advocate - and my (and some list members)
learning more, what would your opinion be if the CI of
the 2 studies did not overlap?</div>
<div><br>
</div>
<div>Appreciate your response.</div>
<div><br>
</div>
<div>Sincerely,</div>
<div>nelly<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, 27 Aug 2020 at
18:21, Gerta Ruecker <<a
href="mailto:ruecker@imbi.uni-freiburg.de"
target="_blank" moz-do-not-send="true">ruecker@imbi.uni-freiburg.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Dear Nelly and all,<br>
<br>
With respect to (only) the first question (sample
size):<br>
<br>
I think nothing is wrong, at least in principle, with
a meta-analysis of <br>
two studies. We analyze single studies, so why not
combining two of <br>
them? They may even include hundreds of patients.<br>
<br>
Of course, it is impossible to obtain a decent
estimate of the <br>
between-study variance/heterogeneity from two or three
studies. But if <br>
the confidence intervals are overlapping, I don't see
any reason to <br>
mistrust the pooled effect estimate.<br>
<br>
Best,<br>
<br>
Gerta<br>
<br>
<br>
<br>
Am 27.08.2020 um 16:07 schrieb ne gic:<br>
> Many thanks for the insights Wolfgang.<br>
><br>
> Apologies for my imprecise questions. By "agreed
upon" & "what<br>
> conclusions/interpretations", I was thinking if
there is a minimum sample<br>
> size whose pooled estimate can be considered
somewhat reliable to produce<br>
> robust inferences e.g. inferences drawn from just
2 studies can be<br>
> drastically changed by the publication of a third
study for instance - but<br>
> it seems like there isn't. But I guess readers
have to then check this for<br>
> themselves to access how much weight they can
place on the conclusions of<br>
> specific meta-analyses.<br>
><br>
> Again, I appreciate it!<br>
><br>
> Sincerely,<br>
> nelly<br>
><br>
> On Thu, Aug 27, 2020 at 3:43 PM Viechtbauer,
Wolfgang (SP) <<br>
> <a
href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl"
target="_blank" moz-do-not-send="true">wolfgang.viechtbauer@maastrichtuniversity.nl</a>>
wrote:<br>
><br>
>> Dear nelly,<br>
>><br>
>> See my responses below.<br>
>><br>
>>> -----Original Message-----<br>
>>> From: R-sig-meta-analysis [mailto:<br>
>> <a
href="mailto:r-sig-meta-analysis-bounces@r-project.org"
target="_blank" moz-do-not-send="true">r-sig-meta-analysis-bounces@r-project.org</a>]<br>
>>> On Behalf Of ne gic<br>
>>> Sent: Wednesday, 26 August, 2020 10:16<br>
>>> To: <a
href="mailto:r-sig-meta-analysis@r-project.org"
target="_blank" moz-do-not-send="true">r-sig-meta-analysis@r-project.org</a><br>
>>> Subject: [R-meta] Sample size and
continuity correction<br>
>>><br>
>>> Dear List,<br>
>>><br>
>>> I have general meta-analysis questions
that are not<br>
>>> platform/software related.<br>
>>><br>
>>> *=======================*<br>
>>> *1. Issue of few included studies *<br>
>>> * =======================*<br>
>>> It seems common to see published
meta-analyses with few studies e.g. :<br>
>>><br>
>>> (A). An analysis of only 2 studies.<br>
>>> (B). In another, subgroup analyses ending
up with only one study in one of<br>
>>> the subgroups.<br>
>>><br>
>>> Nevertheless, they still end up providing
a pooled estimate in their<br>
>>> respective forest plots.<br>
>>><br>
>>> So my question is, is there an agreed
upon (or rule of thumb, or in your<br>
>>> view) minimum number of studies below
which meta-analysis becomes<br>
>>> unacceptable?<br>
>> Agreed upon? Not that I am aware of. Some may
want at least 5 studies (per<br>
>> group or overall), some 10, others may be
fine with if one group only<br>
>> contains 1 or 2 studies.<br>
>><br>
>>> What interpretations/conclusions can one
really draw from such analyses?<br>
>> That's a vague question, so I can't really
answer this in general. Of<br>
>> course, estimates will be imprecise when k is
small (overall or within<br>
>> groups).<br>
>><br>
>>> *===================*<br>
>>> *2. Continuity correction *<br>
>>> * ===================*<br>
>>><br>
>>> In studies of rare events, zero events
tend to occur and it seems common<br>
>> to<br>
>>> add a small value so that the zero is
taken care of somehow.<br>
>>><br>
>>> If for instance, the inclusion of this
small value via continuity<br>
>>> correction leads to differing results
e.g. from non-significant results<br>
>>> when not using correction, to significant
results when using it, what does<br>
>>> make of that? Can we trust such results?<br>
>> If this happens, then the p-value is probably
fluctuating around 0.05 (or<br>
>> whatever cutoff is used for declaring results
as significant). The<br>
>> difference between p=.06 and p=.04 is (very
very unlikely) to be<br>
>> significant (Gelman & Stern, 2006). Or,
to use the words of Rosnow and<br>
>> Rosenthal (1989): "[...] surely, God loves
the .06 nearly as much as the<br>
>> .05".<br>
>><br>
>> Gelman, A., & Stern, H. (2006). The
difference between "significant" and<br>
>> "not significant" is not itself statistically
significant. American<br>
>> Statistician, 60(4), 328-331.<br>
>><br>
>> Rosnow, R.L. & Rosenthal, R. (1989).
Statistical procedures and the<br>
>> justification of knowledge in psychological
science. American Psychologist,<br>
>> 44, 1276-1284.<br>
>><br>
>>> If one instead opts to calculate a risk
difference instead, and test that<br>
>>> for significance, would this be a better
solution (more reliable result?)<br>
>>> to the continuity correction problem
above?<br>
>> If one is worried about the use of
'continuity corrections', then I think<br>
>> the more appropriate reaction is to use
'exact likelihood' methods (such as<br>
>> using (mixed-effects) logistic regression
models or beta-binomial models)<br>
>> instead of switching to risk differences
(nothing wrong with the latter,<br>
>> but risk differences are really a
fudamentally different effect size<br>
>> measure compared to risk/odds ratios).<br>
>><br>
>>> Looking forward to hearing your views as
diverse as they may be in cases<br>
>>> where there is no consensus.<br>
>>><br>
>>> Sincerely,<br>
>>> nelly<br>
> [[alternative HTML version deleted]]<br>
><br>
> _______________________________________________<br>
> R-sig-meta-analysis mailing list<br>
> <a
href="mailto:R-sig-meta-analysis@r-project.org"
target="_blank" moz-do-not-send="true">R-sig-meta-analysis@r-project.org</a><br>
> <a
href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis</a><br>
<br>
-- <br>
<br>
Dr. rer. nat. Gerta Rücker, Dipl.-Math.<br>
<br>
Institute of Medical Biometry and Statistics,<br>
Faculty of Medicine and Medical Center - University of
Freiburg<br>
<br>
Stefan-Meier-Str. 26, D-79104 Freiburg, Germany<br>
<br>
Phone: +49/761/203-6673<br>
Fax: +49/761/203-6680<br>
Mail: <a
href="mailto:ruecker@imbi.uni-freiburg.de"
target="_blank" moz-do-not-send="true">ruecker@imbi.uni-freiburg.de</a><br>
Homepage: <a
href="https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker</a><br>
<br>
_______________________________________________<br>
R-sig-meta-analysis mailing list<br>
<a href="mailto:R-sig-meta-analysis@r-project.org"
target="_blank" moz-do-not-send="true">R-sig-meta-analysis@r-project.org</a><br>
<a
href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis</a><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Dr. rer. nat. Gerta Rücker, Dipl.-Math.
Institute of Medical Biometry and Statistics,
Faculty of Medicine and Medical Center - University of Freiburg
Stefan-Meier-Str. 26, D-79104 Freiburg, Germany
Phone: +49/761/203-6673
Fax: +49/761/203-6680
Mail: <a class="moz-txt-link-abbreviated" href="mailto:ruecker@imbi.uni-freiburg.de">ruecker@imbi.uni-freiburg.de</a>
Homepage: <a class="moz-txt-link-freetext" href="https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker">https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker</a>
</pre>
</body>
</html>