[R] pwr.prop.test and continuity correction

[Frank E Harrell Jr]

Peter Dalgaard wrote:
> Daniel Brewer wrote:
>> Hi,
>>
>> I am trying to sort out a discrepancy between power calculations results
>> between me and another statistician.  I use R but I am not sure what she
>> uses.  It is on the proportions test and so I have been using
>> pwr.prop.test.  I think I have tracked the problem down to pwr.prop.test
>> not using the continuity correction for the test (I did this by using
>> the java applet from
>> http://stat.ethz.ch/R-manual/R-patched/library/stats/html/power.prop.test.html). 
>>
>>
>> So I was wondering whether:
>> 1) Someone could confirm that pwr.prop.test does not use a continuity
>> correction in its calculation.
>> 2) Someone could tell me either how to use pwr.prop.test or another
>> function to get the power of a prop.test with continuity correction.
>> The reason I want this is that I would normally apply the correction
>> when I actually used the test.
>>
>> Many thanks
>>
>> Dan
>>
> 
> power.prop.test (sic) is relying heavily on asymptotic normality, as do 
> similar formulas. It doesn't use continuity correction, but if you're 
> working with such small group sizes, I suspect that the correction term 
> is the least of your worries and that direct simulation would be better.
> 
> (Another source of discrepancy, sometimes seen in textbooks, is that 
> authors use the null variance of p1-p2 also under the alternative. This 
> simplifies the formulas considerably, but it does assume that the actual 
> difference is rather small.)
> 
> R is Open Source. If you want a correction term, it is just a matter of 
> figuring out where to modify expressions like
> 
>     p.body <- quote(pnorm(((sqrt(n) * abs(p1 - p2) -
>         (qnorm(sig.level/tside,
>         lower.tail = FALSE) * sqrt((p1 + p2) * (1 - (p1 +
>          p2)/2))))/sqrt(p1 *
>         (1 - p1) + p2 * (1 - p2)))))
> 
> by adding or subtracting 0.5 or 0.5/n in the appropriate places.
> 
> 

In addition to what Peter said, the continuity correction is in effect 
an attempt to make the proportion test behave like Fisher's exact test 
which is known to be conservative.  We don't usually desire P-values 
that are too large, so I don't recommend the continuity correction.

See the bpower.sim function in the Hmisc package for a simulation-based 
method, and the reference below.

Frank

@Article{cra08how,
   author = 		 {Crans, Gerald G. and Shuster, Jonathan J.},
   title = 		 {How conservative is {Fisher's} exact test? {A} 
quantitative evaluation of the two-sample comparative binomial trial},
   journal = 	 Stat in Med,
   year = 		 2008,
   volume = 	 27,
   pages = 	 {3598-3611},
   annote = 	 {Fisher's exact test; $2\times 2$ contingency table;size 
of test; comparative binomial experiment;first paper to truly quantify 
the conservativeness of Fisher's test;``the test size of FET was less 
than 0.035 for nearly all sample sizes before 50 and did not approach 
0.05 even for sample sizes over 100.'';conservativeness of ``exact'' 
methods}
}


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University