[R] Problem applying McNemar's - Different values in SPSS and R

[Marc Schwartz]

On Dec 28, 2010, at 11:47 AM, Johannes Huesing wrote:

> Marc Schwartz <marc_schwartz at me.com> [Tue, Dec 28, 2010 at 06:30:59PM CET]:
>> 
>> On Dec 28, 2010, at 11:05 AM, Manoj Aravind wrote:
>> 
>>> Hi friends,
>>> I get different values for McNemar's test in R and SPSS. Which one should i
>>> rely on when the p values differ.
> 
> [...]
>> 
>> 
>> The SPSS test appears to be an exact test, whereas the default R function does not perform an exact test, so you are not comparing Apples to Apples...
>> 
> 
> Indeed, binom.test(11, 14) renders the same p-value as SPSS, whereas 
> mcnemar.test() uses the approximation (|a_12 - a_21| - 1)²/(a_21 + a_12) 
> with the "-1" removed if correct=FALSE.
> 
> An old question of mine: Is there any reason not to use binom.test()
> other than historical reasons?


I may be missing the context of your question, but I frequently see exact binomial tests being used when one is comparing the presumptively known probability of some dichotomous characteristic versus that which is observed in an independent sample. For example, in single arm studies where one is comparing an observed event rate against a point estimate for a presumptive historical control.

I also see the use of exact binomial (Clopper-Pearson) confidence intervals being used when one wants to have conservative CI's, given that the nominal coverage of these are at least as large as requested. That is, 95% exact CI's will be at least that large, but in reality can tend to be well above that, depending upon various factors. This is well documented in various papers.

I generally tend to use Wilson CI's for binomial proportions when reporting analyses. I have my own code but these are implemented in various R functions, including Frank's binconf() in Hmisc.

HTH,

Marc