<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>To answer your question, Nelson: <br>
    </p>
    <p>If I have only two studies and the confidence intervals don't
      overlap, I would usually present a forest plot without a pooled
      estimate and discuss this in the text as indication of large
      heterogeneity.</p>
    <p>However, this also depends on the relevance of the difference on
      the outcome scale, depending on subject-matter considerations. For
      example, if I am estimating incidence rate ratios or something
      similar based on very big populations, the CIs may be very short
      and thus non-overlapping, but this may not be important with
      respect to heterogeneity. For an example, see Figure 2b in the
      attached paper (antibiotics density): The first two CIs are not
      overlapping, but this doesn't seems to be a big difference. It's
      only due to the enormous size of the studies.</p>
    <p>Best,</p>
    <p>Gerta<br>
    </p>
    <div class="moz-cite-prefix">Am 27.08.2020 um 18:49 schrieb Nelson
      Ndegwa:<br>
    </div>
    <blockquote type="cite"
cite="mid:CALiySoB17ON3QeZK5QxC922Nsf9TusaKODAfH5u2bCtwe3D5jg@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div>Haha, sorry, I  was editing a response that included your
          signature and forgot to exclude your signature :-)<br>
        </div>
        <div><br>
        </div>
        <div>nelson<br>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Thu, 27 Aug 2020 at 18:47,
          ne gic <<a href="mailto:negic4@gmail.com"
            moz-do-not-send="true">negic4@gmail.com</a>> wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px
          0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div dir="ltr">Wait, are you also Nelly @Nelson?</div>
          <br>
          <div class="gmail_quote">
            <div dir="ltr" class="gmail_attr">On Thu, Aug 27, 2020 at
              6:44 PM Nelson Ndegwa <<a
                href="mailto:nelson.ndegwa@gmail.com" target="_blank"
                moz-do-not-send="true">nelson.ndegwa@gmail.com</a>>
              wrote:<br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
              0.8ex;border-left:1px solid
              rgb(204,204,204);padding-left:1ex">
              <div dir="ltr">
                <div>Dear Gerta,</div>
                <div><br>
                </div>
                <div>I agree with you. In the interest of playing the
                  devil's advocate - and my (and some list members)
                  learning more, what would your opinion be if the CI of
                  the 2 studies did not overlap?</div>
                <div><br>
                </div>
                <div>Appreciate your response.</div>
                <div><br>
                </div>
                <div>Sincerely,</div>
                <div>nelly<br>
                </div>
              </div>
              <br>
              <div class="gmail_quote">
                <div dir="ltr" class="gmail_attr">On Thu, 27 Aug 2020 at
                  18:21, Gerta Ruecker <<a
                    href="mailto:ruecker@imbi.uni-freiburg.de"
                    target="_blank" moz-do-not-send="true">ruecker@imbi.uni-freiburg.de</a>>
                  wrote:<br>
                </div>
                <blockquote class="gmail_quote" style="margin:0px 0px
                  0px 0.8ex;border-left:1px solid
                  rgb(204,204,204);padding-left:1ex">Dear Nelly and all,<br>
                  <br>
                  With respect to (only) the first question (sample
                  size):<br>
                  <br>
                  I think nothing is wrong, at least in principle, with
                  a meta-analysis of <br>
                  two studies. We analyze single studies, so why not
                  combining two of <br>
                  them? They may even include hundreds of patients.<br>
                  <br>
                  Of course, it is impossible to obtain a decent
                  estimate of the <br>
                  between-study variance/heterogeneity from two or three
                  studies. But if <br>
                  the confidence intervals are overlapping, I don't see
                  any reason to <br>
                  mistrust the pooled effect estimate.<br>
                  <br>
                  Best,<br>
                  <br>
                  Gerta<br>
                  <br>
                  <br>
                  <br>
                  Am 27.08.2020 um 16:07 schrieb ne gic:<br>
                  > Many thanks for the insights Wolfgang.<br>
                  ><br>
                  > Apologies for my imprecise questions. By "agreed
                  upon" & "what<br>
                  > conclusions/interpretations", I was thinking if
                  there is a minimum sample<br>
                  > size whose pooled estimate can be considered
                  somewhat reliable to produce<br>
                  > robust inferences e.g. inferences drawn from just
                  2 studies can be<br>
                  > drastically changed by the publication of a third
                  study for instance - but<br>
                  > it seems like there isn't. But I guess readers
                  have to then check this for<br>
                  > themselves to access how much weight they can
                  place on the conclusions of<br>
                  > specific meta-analyses.<br>
                  ><br>
                  > Again, I appreciate it!<br>
                  ><br>
                  > Sincerely,<br>
                  > nelly<br>
                  ><br>
                  > On Thu, Aug 27, 2020 at 3:43 PM Viechtbauer,
                  Wolfgang (SP) <<br>
                  > <a
                    href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl"
                    target="_blank" moz-do-not-send="true">wolfgang.viechtbauer@maastrichtuniversity.nl</a>>
                  wrote:<br>
                  ><br>
                  >> Dear nelly,<br>
                  >><br>
                  >> See my responses below.<br>
                  >><br>
                  >>> -----Original Message-----<br>
                  >>> From: R-sig-meta-analysis [mailto:<br>
                  >> <a
                    href="mailto:r-sig-meta-analysis-bounces@r-project.org"
                    target="_blank" moz-do-not-send="true">r-sig-meta-analysis-bounces@r-project.org</a>]<br>
                  >>> On Behalf Of ne gic<br>
                  >>> Sent: Wednesday, 26 August, 2020 10:16<br>
                  >>> To: <a
                    href="mailto:r-sig-meta-analysis@r-project.org"
                    target="_blank" moz-do-not-send="true">r-sig-meta-analysis@r-project.org</a><br>
                  >>> Subject: [R-meta] Sample size and
                  continuity correction<br>
                  >>><br>
                  >>> Dear List,<br>
                  >>><br>
                  >>> I have general meta-analysis questions
                  that are not<br>
                  >>> platform/software related.<br>
                  >>><br>
                  >>> *=======================*<br>
                  >>> *1. Issue of few included studies *<br>
                  >>> * =======================*<br>
                  >>> It seems common to see published
                  meta-analyses with few studies e.g. :<br>
                  >>><br>
                  >>> (A). An analysis of only 2 studies.<br>
                  >>> (B). In another, subgroup analyses ending
                  up with only one study in one of<br>
                  >>> the subgroups.<br>
                  >>><br>
                  >>> Nevertheless, they still end up providing
                  a pooled estimate in their<br>
                  >>> respective forest plots.<br>
                  >>><br>
                  >>> So my question is, is there an agreed
                  upon (or rule of thumb, or in your<br>
                  >>> view) minimum number of studies below
                  which meta-analysis becomes<br>
                  >>> unacceptable?<br>
                  >> Agreed upon? Not that I am aware of. Some may
                  want at least 5 studies (per<br>
                  >> group or overall), some 10, others may be
                  fine with if one group only<br>
                  >> contains 1 or 2 studies.<br>
                  >><br>
                  >>> What interpretations/conclusions can one
                  really draw from such analyses?<br>
                  >> That's a vague question, so I can't really
                  answer this in general. Of<br>
                  >> course, estimates will be imprecise when k is
                  small (overall or within<br>
                  >> groups).<br>
                  >><br>
                  >>> *===================*<br>
                  >>> *2. Continuity correction *<br>
                  >>> * ===================*<br>
                  >>><br>
                  >>> In studies of rare events, zero events
                  tend to occur and it seems common<br>
                  >> to<br>
                  >>> add a small value so that the zero is
                  taken care of somehow.<br>
                  >>><br>
                  >>> If for instance, the inclusion of this
                  small value via continuity<br>
                  >>> correction leads to differing results
                  e.g. from non-significant results<br>
                  >>> when not using correction, to significant
                  results when using it, what does<br>
                  >>> make of that? Can we trust such results?<br>
                  >> If this happens, then the p-value is probably
                  fluctuating around 0.05 (or<br>
                  >> whatever cutoff is used for declaring results
                  as significant). The<br>
                  >> difference between p=.06 and p=.04 is (very
                  very unlikely) to be<br>
                  >> significant (Gelman & Stern, 2006). Or,
                  to use the words of Rosnow and<br>
                  >> Rosenthal (1989): "[...] surely, God loves
                  the .06 nearly as much as the<br>
                  >> .05".<br>
                  >><br>
                  >> Gelman, A., & Stern, H. (2006). The
                  difference between "significant" and<br>
                  >> "not significant" is not itself statistically
                  significant. American<br>
                  >> Statistician, 60(4), 328-331.<br>
                  >><br>
                  >> Rosnow, R.L. & Rosenthal, R. (1989).
                  Statistical procedures and the<br>
                  >> justification of knowledge in psychological
                  science. American Psychologist,<br>
                  >> 44, 1276-1284.<br>
                  >><br>
                  >>> If one instead opts to calculate a risk
                  difference instead, and test that<br>
                  >>> for significance, would this be a better
                  solution (more reliable result?)<br>
                  >>> to the continuity correction problem
                  above?<br>
                  >> If one is worried about the use of
                  'continuity corrections', then I think<br>
                  >> the more appropriate reaction is to use
                  'exact likelihood' methods (such as<br>
                  >> using (mixed-effects) logistic regression
                  models or beta-binomial models)<br>
                  >> instead of switching to risk differences
                  (nothing wrong with the latter,<br>
                  >> but risk differences are really a
                  fudamentally different effect size<br>
                  >> measure compared to risk/odds ratios).<br>
                  >><br>
                  >>> Looking forward to hearing your views as
                  diverse as they may be in cases<br>
                  >>> where there is no consensus.<br>
                  >>><br>
                  >>> Sincerely,<br>
                  >>> nelly<br>
                  >       [[alternative HTML version deleted]]<br>
                  ><br>
                  > _______________________________________________<br>
                  > R-sig-meta-analysis mailing list<br>
                  > <a
                    href="mailto:R-sig-meta-analysis@r-project.org"
                    target="_blank" moz-do-not-send="true">R-sig-meta-analysis@r-project.org</a><br>
                  > <a
                    href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis"
                    rel="noreferrer" target="_blank"
                    moz-do-not-send="true">https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis</a><br>
                  <br>
                  -- <br>
                  <br>
                  Dr. rer. nat. Gerta Rücker, Dipl.-Math.<br>
                  <br>
                  Institute of Medical Biometry and Statistics,<br>
                  Faculty of Medicine and Medical Center - University of
                  Freiburg<br>
                  <br>
                  Stefan-Meier-Str. 26, D-79104 Freiburg, Germany<br>
                  <br>
                  Phone:    +49/761/203-6673<br>
                  Fax:      +49/761/203-6680<br>
                  Mail:     <a
                    href="mailto:ruecker@imbi.uni-freiburg.de"
                    target="_blank" moz-do-not-send="true">ruecker@imbi.uni-freiburg.de</a><br>
                  Homepage: <a
href="https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker"
                    rel="noreferrer" target="_blank"
                    moz-do-not-send="true">https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker</a><br>
                  <br>
                  _______________________________________________<br>
                  R-sig-meta-analysis mailing list<br>
                  <a href="mailto:R-sig-meta-analysis@r-project.org"
                    target="_blank" moz-do-not-send="true">R-sig-meta-analysis@r-project.org</a><br>
                  <a
                    href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis"
                    rel="noreferrer" target="_blank"
                    moz-do-not-send="true">https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis</a><br>
                </blockquote>
              </div>
            </blockquote>
          </div>
        </blockquote>
      </div>
    </blockquote>
    <pre class="moz-signature" cols="72">-- 

Dr. rer. nat. Gerta Rücker, Dipl.-Math.

Institute of Medical Biometry and Statistics,
Faculty of Medicine and Medical Center - University of Freiburg  

Stefan-Meier-Str. 26, D-79104 Freiburg, Germany

Phone:    +49/761/203-6673
Fax:      +49/761/203-6680
Mail:     <a class="moz-txt-link-abbreviated" href="mailto:ruecker@imbi.uni-freiburg.de">ruecker@imbi.uni-freiburg.de</a>
Homepage: <a class="moz-txt-link-freetext" href="https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker">https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker</a>
</pre>
  </body>
</html>