<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=windows-1252">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<pre><!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:DoNotShowPropertyChanges/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:DontVertAlignCellWithSp/>
<w:DontBreakConstrainedForcedTables/>
<w:DontVertAlignInTxbx/>
<w:Word11KerningPairs/>
<w:CachedColBalance/>
</w:Compatibility>
<w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]-->
</pre>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">Dear Dirk,</span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">I came accross this communication
by chance when doing an internet search for something else, but
nevertheless thought it prudent to respond albeit after some
time has passed. <br>
</span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">I would disagree with the comments
made by Wolfgang and I believe the history is also not that
accurate.</span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">First, the IVhet model is not a
random-effects model with
1/vi weights. It is a fixed effect model with robust error
variance.
Unfortunately statisticians are unable to easily get away from
the concept of
normally distributed random effects and the IVhet model is based
on the fact
that there is an underlying parameter from which all study
effects emerge with
some forms of error (which I will not discuss here for brevity).</span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">Second, the IVhet model was first
described as the quality
effects model in 2008 (well before Henmi and Copas). Indeed, the
IVhet model IS
the quality effects model with one constraint – all quality is
set to equal so
no quality input is required. To check this run the QE model in
metaXL and enter
quality (any acceptable value) against each study that is the
same for all
studies and you will get the IVhet result. The reason IVhet was
created was
that we realized in the fifth year after the QE release that
biostatisticians
were simply unable to envisage quality in the way we do and this
was a way to
bring it back without mentioning quality – this has worked very
well as though
we lose bias adjustment, we still have a much better estimator
than the RE
estimator in terms of MSE and coverage and is much more utilised
than the QE estimator in research.<br>
</span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">Third, Henmi and Copas propose a CI
that is not really optimal since it is wider than the IVhet/QE
intervals and the latter are known to have at least nominal
coverage</span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">Fourth, given that this is a fixed
effect model, the use of
tau squared is as an overdispersion correction and thus ONLY the
DL tau squared
defines the IVhet and QE models. Many biostatisticians have
tried to replace
the DL tau squared with REML and other variants – these are no
longer IVhet or
QE models because the conceptualization of the IVhet model
depends on the fact
that tau squared be used as an overdispersion correction and
thus must be generated
via method of moments. While the variance formulation provided
by Wolfgang in
the slides works, it will fail as soon as a different tau
squared (more
accurate in the biostatistics parlance) is inserted <span
style="mso-spacerun:yes"> </span><span
style="mso-spacerun:yes"> </span>The
fact that this formulation has been put forward now is in my
view an ex post
facto justification from biostatistics for missing this easily
conceptualized estimator
in the first place and despite the fact that it is far superior
to RE
estimators, nothing much has changed. </span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman""><span style="mso-spacerun:yes"> </span>Finally,
a recent paper
by Rice et al has added more confusion to this area by
distinguishing fixed effect (singular)
and fixed effects (plural) models. My view is that there are
neither fixed
effects (plural) nor random effects models and these are just
attempts by
biostatistics to fit models based on their worldview, the key
element of this
worldview being the existence of normally distributed random
effects. Such
models have survived simulation testing because most researchers
tend to
simulate the way they will eventually analyse thus creating a
self-fulfilling
prophecy. </span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman"">Apologies if my comments seem a bit
harsh but I was a
clinician working in the hospital for 20 years and an ardent
user of research
syntheses. I took up clinical epidemiology when I realized that
there was a serious
problem with research synthesis that needed to be fixed from
outside of
mainstream biostatistics and after discussions with eminent
biostatisticians
failed to generate much change.. <span style="mso-spacerun:yes"> </span><span
style="mso-spacerun:yes"> </span><span
style="mso-spacerun:yes"> </span></span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman""><span style="mso-spacerun:yes">Best</span></span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman""><span style="mso-spacerun:yes">Suhail</span></span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-font-family:
"Times New Roman""><span style="mso-spacerun:yes"><br>
</span></span></p>
<p class="MsoNormal"
style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal;tab-stops:45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt
320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt
687.0pt 732.8pt"><span style="font-size:13.5pt">Suhail A. R. Doi<br>
Professor of Clinical Epidemiology (Hon),<br>
Research School of Population Health<br>
ANU College of Health and Medicine, <br>
62 Mills Rd<br>
The Australian National University<br>
Acton ACT 2601<br>
E: <a class="moz-txt-link-abbreviated" href="mailto:Suhail.Doi@anu.edu.au">Suhail.Doi@anu.edu.au</a><br>
<br>
CRICOS Provider # 00120C</span><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";
mso-fareast-font-family:"Times New Roman""></span></p>
<br>
<br>
<pre><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0in;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}
</style>
<![endif]-->_______________________________________________________________
As far as I am concerned, discussions around the pros and cons of various methods are perfectly fine, esp. if they are directly linked to implementations in R.
So, we are considering two methods:
1) A random-effects model with the standard 1/(vi + tau^2) weights (where vi is the sampling variance of the ith study and tau^2 the (estimated) amount of variance/heterogeneity in the true outcomes)
versus
2) A random-effects model with 1/vi weights.
Under the assumptions of the RE model and in the absence of publication bias, both approaches provide an unbiased estimate of the average true outcome. Approach 1 is more efficient; in fact, using 1/(vi + tau^2) weights gives us the uniformly minimum variance unbiased estimator (UMVUE).
Sidenote: To be precise, that is only true if tau^2 would be a known quantity and not estimated (and similarly, the sampling variances must be known quantities). So, really, we are only getting an approximation to the UMVUE. The larger k (number of studies) is, the more appropriate it is to treat tau^2 as a known quantity. The larger the within-study sample sizes are, the more appropriate it is to treat the sampling variances as known quantities (but what 'large' means here depends a lot on the outcome measure used; for measures based on a variance-stabilizing transformation, even rather small within-study sample sizes will do).
Things become complicated when there is publication bias, that it, when the probability of including a study in our meta-analysis is tied to the statistical significance of the finding/outcome. In that case, large studies (with very small sampling variances) will provide less biased estimates than small studies (with very large sampling variances). Now if tau^2 is very large, then tau^2 dominates the 1/(vi + tau^2) weights, so that all studies get almost the same weight, and hence also the very small studies that are so biased. As a result, the estimate of the average true outcome will also be badly biased. If, instead, we use 1/vi weights, then the very small studies are downweighted a lot and don't screw up our estimate as much.
That is in essence what Henmi and Copas (2010) have shown. They also derived a method to compute the SE/CI for approach 2, which is implemented in the hc() function in metafor. To go back to the earlier example:
library(metafor)
dat <- get(data(dat.li2007))
dat <- dat[order(dat$study),]
rownames(dat) <- 1:nrow(dat)
dat <- escalc(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i, data=dat, subset=-c(19,21,22))
### standard RE model
res <- rma(yi, vi, data=dat, method="DL")
predict(res, transf=exp, digits=2)
### Henmi & Copas (2010) method
hc(res, transf=exp, digits=2)
### RE model with 1/vi weights ("IVhet")
res <- rma(yi, vi, data=dat, method="DL", weights=1/vi)
predict(res, transf=exp, digits=2)
Interestingly, the H&C method gives a MUCH wider CI here. Usually though, the difference between the H&C method and the "rma(yi, vi, data=dat, method="DL", weights=1/vi)" approach ("IVhet") is rather small.
I gave a talk at the 2016 meeting of the Society for Research Synthesis Methodology about this topic. The slides are here in case you are interested:
<a href="http://www.wvbauer.com/lib/exe/fetch.php/talks:2016_viechtbauer_srsm_weights.pdf">http://www.wvbauer.com/lib/exe/fetch.php/talks:2016_viechtbauer_srsm_weights.pdf</a>
In the simulation, I also compared the H&C method with the "IVhet" approach (not shown in the slides) and found that the H&C approach did just a tad better, but not by much. A disadvantage of the H&C approach is that it doesn't generalize to models with moderators (meta-regression).
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis-bounces at r-project.org</a>] On Behalf Of <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">dirk.richter at upd.unibe.ch</a>
Sent: Wednesday, 29 November, 2017 22:29
To: <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis at r-project.org</a>
Subject: Re: [R-meta] Implementation of the Inverse variance heterogeneity model
Dear Wolfgang, dear James,
many thanks to both of you for replying so quickly and for providing some valuable history lessons to an R-meta newbie.
My next question may be a bit off topic as it is more general on the pros and cons of these different approaches (am I allowed to put it here??). At first sight, the inverse sampling variance approach and those you have mentioned have an appeal to me, especially when comparing them to conventional RE and its use of rather similar weights of larger and smaller samples. The newbie that I am would like to have some guidance on these issues or at least an in-depth discussion paper. Does anybody have a recommendation?
Thanks,
Dirk
Von: James Pustejovsky [mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">jepusto at gmail.com</a>]
Gesendet: Mittwoch, 29. November 2017 21:02
An: Viechtbauer Wolfgang (SP) <<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">wolfgang.viechtbauer at maastrichtuniversity.nl</a>>
Cc: Richter, Dirk (UPD) <<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">dirk.richter at upd.unibe.ch</a>>; <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis at r-project.org</a>
Betreff: Re: [R-meta] Implementation of the Inverse variance heterogeneity model
I was just typing up an email saying the same thing (and using the same example), but Wolfgang beat me to the punch! So count it as independently replicated. I would add two things:
1. An alternative to the IVhet method is to use the FE model with robust variance estimation (Sidik & Jonkman, 2006) to account for between-study heterogeneity when estimating standard errors. This can be done with the clubSandwich package (though you'll have to do the scale transformation as a post-processing step):
### standard FE model
res <- rma(yi, vi, data=dat, method="FE")
library(clubSandwich)
coef_test(res, vcov = "CR2", cluster = dat$id)
In this example, the robust standard error is *substantially* smaller than the IVhet standard error. It also has very low degrees of freedom because of the very unequal weighting of the studies.
2. In the conventional random effects model, the Knapp-Hartung method is often recommended for testing the average treatment effect:
### standard RE model with Knapp-Hartung
res <- rma(yi, vi, data=dat, method="DL", test = "knha")
predict(res, transf=exp, digits=2)
I don't know if there is research into the relative performance of Knapp-Hartung with inverse-sampling variance weights (anybody know of work on this?), but on the face of it, it seems reasonable to generalize based on its performance under conventional RE models:
### RE model with 1/vi weights ("IVhet")
res <- rma(yi, vi, data=dat, method="DL", weights=1/vi, test = "knha")
predict(res, transf=exp, digits=2)
James
On Wed, Nov 29, 2017 at 1:42 PM, Viechtbauer Wolfgang (SP) <<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">wolfgang.viechtbauer at maastrichtuniversity.nl</a><mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">wolfgang.viechtbauer at maastrichtuniversity.nl</a>>> wrote:
Dear Dirk,
What Doi et al. describe are RE models with different weights than the default ones.
"AMhet" uses unit weights. The possibility to fit this model was implemented in metafor since its first release in 2009. "IVhet" uses inverse sampling variance weights. The possibility to fit this model was implemented in version 1.9-3 in 2014.
Using the example from Doi et al. (2015):
##############################
library(metafor)
dat <- get(data(dat.li2007))
dat <- dat[order(dat$study),]
rownames(dat) <- 1:nrow(dat)
dat <- escalc(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i, data=dat, subset=-c(19,21,22))
### standard RE model
res <- rma(yi, vi, data=dat, method="DL")
predict(res, transf=exp, digits=2)
### RE model with 1/vi weights ("IVhet")
res <- rma(yi, vi, data=dat, method="DL", weights=1/vi)
predict(res, transf=exp, digits=2)
### RE model with unit weights ("AMhet")
res <- rma(yi, vi, data=dat, method="DL", weights=1)
predict(res, transf=exp, digits=2)
##############################
The results are exactly those reported on 135: "When the meta-analytic estimates were computed using the three methods, they were most extreme with the AMhet estimator (OR 0.44; 95% CI 0.29-0.66), less extreme with the RE estimator (OR 0.71; 95% CI 0.57-0.89) and most conservative with the IVhet estimator (OR 1.01; 95% CI 0.71-1.46)."
The idea to fit a RE model with inverse sampling variance weights was actually already described in:
Henmi, M., & Copas, J. B. (2010). Confidence intervals for random effects meta-analysis and robustness to publication bias. Statistics in Medicine, 29(29), 2969-2983.
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis-bounces at r-project.org</a><mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis-bounces at r-project.org</a>>] On Behalf Of <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">dirk.richter at upd.unibe.ch</a><mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">dirk.richter at upd.unibe.ch</a>>
Sent: Wednesday, 29 November, 2017 17:22
To: <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis at r-project.org</a><mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">r-sig-meta-analysis at r-project.org</a>>
Subject: [R-meta] Implementation of the Inverse variance heterogeneity model
Dear R meta-analysis group,
I was wondering whether there are any plans to implement the Inverse variance heterogeneity model (by Doi et al., see reference below) into R MA packages or whether this has been done recently (although I couldn't find anything on the Web). While the authors of this model have provided with MetaXL a free software that allows to run such an analysis, I would be happy to have it connected to or implemented into R to have the chance to run meta-regressions based on this approach. Currently, there is a only a connection to Stata for meta-regressions.
Reference
SA Doi et al. Advances in the meta-analysis of heterogeneous clinical trials I: The inverse variance heterogeneity model. Contemp Clin Trials. 2015 Nov;45(Pt A):130-8. doi: 10.1016/j.cct.2015.05.009
Thanks,
Dirk Richter
UNIVERSITÄRE PSYCHIATRISCHE DIENSTE BERN (UPD) AG
DIREKTION PSYCHIATRISCHE REHABILITATION
Dirk Richter, Dr. phil. habil.
Leiter Forschung und Entwicklung
Murtenstrasse 46
CH-3008 Bern
Tel. +41 31 632 4707
Mobil + 41 76 717 5220
E-Mail: <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">dirk.richter at upd.unibe.ch</a><mailto:<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis">dirk.richter at upd.unibe.ch</a>>
<a href="https://www.upd.ch/forschung/psychiatrische-rehabilitation/">https://www.upd.ch/forschung/psychiatrische-rehabilitation/</a>
University of Bern Psychiatric Services
Center for Psychiatric Rehabilitation
Dirk Richter, Dr. phil., PhD
Head of Research and Development
Murtenstrasse 46
CH-3008 Bern
Switzerland
Phone +41 31 632 4707<a class="moz-txt-link-rfc2396E" href="tel:%2B41%2031%20632%204707"><tel:%2B41%2031%20632%204707></a>
Mobile Phone +41 76 717 5220<<a class="moz-txt-link-freetext" href="tel:%2B41%2">tel:%2B41%2</a></pre>
</body>
</html>