<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link="#0563C1" vlink="#954F72" style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal>No-one has responded to this issue. It's now causing a problem in my simulations when I am analyzing for publication bias arising from deletion of 90% of nonsignificant study estimates and ending up with small numbers (10-30) of included studies. See below (and attached as an easier-to-read text file) for an example. Two of the 14 study estimates (Row 8 and 9) were non-significant, but the original t value (tOrig) would have made them significant in selmodel(…, type = "step", steps = (0.025)). So I processed any observations with non-significant p values and t>1.96 by replacing the standard error (YdelSE) with Ydelta/1.95. The resulting new t vslues (tNew) are 1.95 for both those observations, whereas all the other t values are unchanged. So they should be non-significant in selmodel, right? But I still get this error message:<o:p></o:p></p><p class=MsoNormal>Error in selmodel.rma.uni(x, type = "step", steps = (0.025)) : <o:p></o:p></p><p class=MsoNormal> One or more intervals do not contain any observed p-values (use 'verbose=TRUE' to see which).<o:p></o:p></p><p class=MsoNormal>I must be doing something idiotic, but what? Help, please!<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Oh, and thanks again to Tobias Saueressig for his help with list-processing of the objects created by rma, selmodel and confint. My original for-loop approach fell over when the values of the Sim variable were not consecutive integers (for example, when I had generated the sims and then deleted any lacking non-significant study estimates), but separate processing of the lists as suggested by Tobias worked perfectly. It stops working when it crashes out with the above error, but hopefully someone will solve that problem.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Will<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> Sim StudID Sex SSize Ydelta YdelSE tOrig tNew pValue<o:p></o:p></p><p class=MsoNormal> <dbl> <dbl> <fct> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl><o:p></o:p></p><p class=MsoNormal>1 448 1 Female 10 3.72 0.684 5.44 5.44 0.000413<o:p></o:p></p><p class=MsoNormal>2 448 6 Female 10 3.08 0.901 3.42 3.42 0.00766<o:p></o:p></p><p class=MsoNormal>3 448 11 Female 10 4.49 0.926 4.85 4.85 0.000906<o:p></o:p></p><p class=MsoNormal>4 448 21 Female 28 4.95 0.777 6.37 6.37 0.000000808<o:p></o:p></p><p class=MsoNormal>5 448 26 Female 12 3.82 1.25 3.06 3.06 0.0109<o:p></o:p></p><p class=MsoNormal>6 448 31 Female 22 2.13 0.991 2.15 2.15 0.0433<o:p></o:p></p><p class=MsoNormal>7 448 36 Female 10 3.27 1.13 2.89 2.89 0.0177<o:p></o:p></p><p class=MsoNormal>8 448 10 Male 18 4.46 2.29 2.03 1.95 0.0578<o:p></o:p></p><p class=MsoNormal>9 448 14 Male 10 3.2 1.64 1.98 1.95 0.0795<o:p></o:p></p><p class=MsoNormal>10 448 17 Male 13 4.32 1.97 2.19 2.19 0.049<o:p></o:p></p><p class=MsoNormal>11 448 30 Male 10 1.16 0.467 2.48 2.48 0.0348<o:p></o:p></p><p class=MsoNormal>12 448 38 Male 10 3.61 1.24 2.91 2.91 0.0175<o:p></o:p></p><p class=MsoNormal>13 448 39 Male 10 2.49 0.828 3.01 3.01 0.0148<o:p></o:p></p><p class=MsoNormal>14 448 40 Male 28 1.92 0.602 3.19 3.19 0.0036<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-family:"Calibri",sans-serif;mso-ligatures:none'>From:</span></b><span style='font-family:"Calibri",sans-serif;mso-ligatures:none'> Will Hopkins <willthekiwi@gmail.com> <br><b>Sent:</b> Friday, March 15, 2024 8:39 AM<br><b>To:</b> 'R Special Interest Group for Meta-Analysis' <r-sig-meta-analysis@r-project.org><br><b>Subject:</b> Calculation of p values in selmodel<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>According to your documentation, Wolfgang, the selection models in selmodel are based on the p values of the study estimates, but these are computed by assuming the study estimate divided by its standard error has a normal distribution, whereas significance in the original studies of mean effects of continuous variables would have been based on a t distribution. It could make a difference when sample sizes in the original studies are ~10 or so, because some originally non-significant effects would be treated as significant by selmodel. For example, with a sample size of 10, a mean change has 9 degrees of freedom, so a p value of 0.080 (i.e., non-significant, p>0.05) in the original study will be given a p value of 0.049 (i.e., significant, p<0.05) by selmodel. Is this issue likely to make any real difference to the performance of selmodel with meta-analyses of realistic small-sample studies? I guess that only a small (negligible?) proportion of p values will fall between 0.05 and 0.08, in the worst-case scenario of a true effect close to the critical value and with only 9 degrees of freedom for the SE. If it is an issue, you could include the SE's degrees of freedom in the rma object that gets passed to selmodel.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Will<o:p></o:p></p></div></body></html>