[Rd] power.prop.test() documentation question

[Marc Schwartz]

Hi All,

Based upon a discussion on power/sample size calculations on another, non-R related, list, some light bulbs went on regarding the assumptions of what type of statistical test is going to be used with various power/sample size calculators/functions for proportions. In some cases, this is clearly stated, in others, it is not.

In the case of power.prop.test() and comparing outputs against other calculators, there appears to be an implied presumption that an un-corrected chi-square test will be used, as opposed to a corrected chi-square or Fisher Exact Test (FET), in the 2x2 case. Sample sizes for the un-corrected chi-square will generally be smaller than either the corrected chi-square or the FET, given similar inputs, where the latter two, not surprisingly given their common conservative bias, will yield similar sample size results. 

This is not explicitly documented in ?power.prop.test, though it is in some other applications, as noted above. 

As a particular example from the other discussions, using p1 = 0.142, p2 = 0.266, with power = 0.8 and sig.level = 0.05, power.prop.test() yields a sample size of ~165 per group. Other calculators that presume either a corrected chi-square or the FET, yield ~180 per group. 

I raise this issue, as should one use the function to calculate a prospective sample size for a study, and then actually use a corrected chi-square to analyze the data, per routine use and/or a formal analysis plan, the power of that test will be lower than that which was presumed for the a priori calculation. It may not make a big difference in some proportion of the cases relative to p <= alpha, but given the idiosyncrasies of the observed data at the end of the study, along with the effective loss of some power, it may very well be relevant to the results and their strict interpretation. It may also impact, to some extent, the a priori planning for the study, relative to the needed target sample size, budgeting and other considerations for a study sponsor.

Is there any logic in adding some notes to ?power.prop.test, to indicate the implied presumption of the use of an un-corrected chi-square test? 

Thanks for any comments, including telling me that I need more caffeine and to increase my oxygen uptake...

Regards,

Marc Schwartz