[R] Can I use "mcnemar.test" for 3*3 tables (or is there a bug in the command?)

David Winsemius dwinsemius at comcast.net
Mon Jul 20 02:20:20 CEST 2009


On Jul 19, 2009, at 6:09 PM, Tal Galili wrote:

> Hello Charles,
> Thank you for the detail reply.
>
> I am still left with the leading question which is: which test  
> should I use
> when analyzing the 3 by 3 matrix I have? The mcnemar.test or the   
> mh_test?
> Is the one necessarily better then the other?

Please define "better".

> (for example for
> sparser matrices ?)

That does not help.

>
> What about:
> mh_test(as.table(matrix(1:16,4)))
> It returns a very significant result:
> chi-squared = 11.4098, df = 3, p-value = 0.009704
>
> Where as "mcnemar.test(matrix(1:16,4))", didn't:
> McNemar's chi-squared = 11.5495, df = 6, p-value = 0.0728
>
> So which one is "right" ?

And now ... define "right".

> (from the looks of it, the mh_test is doing much better)

Perhaps from the perspective of a statistically naive reviewer.

>
> Should the strategy be to try and use both methods, and start  
> digging when
> one doesn't sit well with the other?

I am reminded of Jim Holtam's tag line:  "What problem are you trying  
to solve?"

>
>
> Thanks,
> Tal
>
>
>
> On Sun, Jul 19, 2009 at 10:26 PM, Charles C. Berry <cberry at tajo.ucsd.edu 
> >wrote:
>
>> On Sun, 19 Jul 2009, Tal Galili wrote:
>>
>> Hello David,Thank you for your answer.
>>>
>>> Do you know then what does the "mcnemar.test" do in the case of a  
>>> 3*3
>>> table
>>> ?
>>>
>>
>>       print(mcnemar.test)
>>
>> will show you what it does.
>>
>> Because the results for the simple example I gave are rather  
>> different (P
>>> value of 0.053 VS 0.73)
>>>
>>
>> The test mcnemar.test() constructs is one of symmetry, which is  
>> equivalent
>> to marginal homogenity in hierarchical log-linear models as I  
>> recall from
>> Bishop, Fienberg, and Holland's 1975 opus on count data.
>>
>> Stuart-Maxwell uses the dispersion matrix of marginal difference.
>>
>> These are two different tests. I suspect that Stuart-Maxwell is less
>> susceptible to continuity issues in very sparse tables, which may  
>> account
>> for the difference you see here.
>>
>>
>>
>>> In case the mcnemar can't really handle a 3*3 matrix (or more),  
>>> shouldn't
>>> there be an error massage for this case? (if so, who should I turn  
>>> to, in
>>> order to report this?)
>>>
>>
>> Well, the code is pretty straightforward and
>>
>>       mcnemar.test(matrix(1:16,4))
>>
>> returns 11.5495 which is correct.
>>
>> It looks like there is nothing to report. 3,1,5), ncol = 3))))
>>
>>
>> Chuck
>>
>>
>>> Thanks again,
>>> Tal
>>>
>>>
>>>
>>>
>>>
>>> On Sun, Jul 19, 2009 at 3:47 PM, David Freedman  
>>> <3.14david at gmail.com>
>>> wrote:
>>>
>>>
>>>> There is a function mh_test in the coin package.
>>>>
>>>> library(coin)
>>>> mh_test(tt)
>>>>
>>>> The documentation states, "The null hypothesis of independence of  
>>>> row and
>>>> column totals is tested. The corresponding test for binary  
>>>> factors x and
>>>> y
>>>> is known as McNemar test. For larger tables, Stuart’s W0 statistic
>>>> (Stuart,
>>>> 1955, Agresti, 2002, page 422, also known as Stuart-Maxwell test)  
>>>> is
>>>> computed."
>>>>
>>>> hth, david freedman
>>>>
>>>>
>>>> Tal Galili wrote:
>>>>
>>>>>
>>>>> Hello all,
>>>>>
>>>>> I wish to perform a mcnemar test for a 3 by 3 matrix.
>>>>> By running the slandered R command I am getting a result but I  
>>>>> am not
>>>>>
>>>> sure
>>>>
>>>>> I
>>>>> am getting the correct one.
>>>>> Here is an example code:
>>>>>
>>>>> (tt <-  as.table(t(matrix(c(1,4,1    ,
>>>>>                           0,5,5,
>>>>>                           3,1,5), ncol = 3))))
>>>>> mcnemar.test(tt, correct=T)
>>>>> #And I get:
>>>>>       McNemar's Chi-squared test
>>>>> data:  tt
>>>>> McNemar's chi-squared = 7.6667, df = 3, p-value = *0.05343*
>>>>>
>>>>>
>>>>> Now I was wondering if the test I just performed is the correct  
>>>>> one.
>>>>>
>>>>>> From looking at the Wikipedia article on mcnemar (
>>>>>>
>>>>> http://en.wikipedia.org/wiki/McNemar's_test), it is said that:
>>>>> "The Stuart-Maxwell
>>>>> test<http://ourworld.compuserve.com/homepages/jsuebersax/mcnemar.htm 
>>>>> >
>>>>> is
>>>>> different generalization of the McNemar test, used for testing  
>>>>> marginal
>>>>> homogeneity in a square table with more than two rows/columns"
>>>>>
>>>>> From searching for a Stuart-Maxwell
>>>>>>
>>>>> test<http://ourworld.compuserve.com/homepages/jsuebersax/mcnemar.htm 
>>>>> >
>>>>> in
>>>>> google, I found an algorithm here:
>>>>>
>>>>>
>>>> http://www.m-hikari.com/ams/ams-password-2009/ams-password9-12-2009/abbasiAMS9-12-2009.pdf
>>>>
>>>>>
>>>>> From running this algorithm I am getting a different P value,  
>>>>> here is
>>>>>> the
>>>>>>
>>>>> (somewhat ugly) code I produced for this:
>>>>> get.d <- function(xx)
>>>>> {
>>>>> length1 <- dim(xx)[1]
>>>>> ret1 <- margin.table(xx,1) - margin.table(xx,2)
>>>>> return(ret1)
>>>>> }
>>>>>
>>>>> get.s <- function(xx)
>>>>> {
>>>>> the.s <- xx
>>>>> for( i in 1:dim(xx)[1])
>>>>> {
>>>>>   for(j in 1:dim(xx)[2])
>>>>>   {
>>>>>     if(i == j)
>>>>>     {
>>>>>       the.s[i,j] <- margin.table(xx,1)[i] + margin.table(xx,2) 
>>>>> [i] -
>>>>> 2*xx[i,i]
>>>>>     } else {
>>>>>       the.s[i,j] <- -(xx[i,j] + xx[j,i])
>>>>>     }
>>>>>   }
>>>>> }
>>>>> return(the.s)
>>>>> }
>>>>>
>>>>> chi.statistic <- t(get.d(tt)[-3]) %*% solve(get.s(tt)[-3,-3])  %*%
>>>>> get.d(tt)[-3]
>>>>> paste("the P value:", pchisq(chi.statistic, 2))
>>>>>
>>>>> #and the result was:
>>>>> "the P value: 0.268384371053358"
>>>>>
>>>>>
>>>>>
>>>>> So to summarize my questions:
>>>>> 1) can I use "mcnemar.test" for 3*3 (or more) tables ?
>>>>> 2) if so, what test is being performed (
>>>>> Stuart-Maxwell<
>>>>>
>>>> http://ourworld.compuserve.com/homepages/jsuebersax/mcnemar.htm>)
>>>>
>>>>> ?
>>>>> 3) Do you have a recommended link to an explanation of the  
>>>>> algorithm
>>>>> employed?
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Tal
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> ----------------------------------------------
>>>>>
>>>>>
>>>>> My contact information:
>>>>> Tal Galili
>>>>> Phone number: 972-50-3373767
>>>>> FaceBook: Tal Galili
>>>>> My Blogs:
>>>>> http://www.r-statistics.com/
>>>>> http://www.talgalili.com
>>>>> http://www.biostatistics.co.il
>>>>>
>>>>>     [[alternative HTML version deleted]]
>>>>>
>>>>> ______________________________________________
>>>>> R-help at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>> PLEASE do read the posting guide
>>>>> http://www.R-project.org/posting-guide.html
>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>
>>>>>
>>>>>
>>>> --
>>>> View this message in context:
>>>>
>>>> http://www.nabble.com/Can-I-use-%22mcnemar.test%22-for-3*3-tables-%28or-is-there-a-bug-in-the-command-%29-tp24556414p24556693.html
>>>> Sent from the R help mailing list archive at Nabble.com.
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide
>>>> http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>>
>>>
>>>
>>> --
>>> ----------------------------------------------
>>>
>>>
>>> My contact information:
>>> Tal Galili
>>> Phone number: 972-50-3373767
>>> FaceBook: Tal Galili
>>> My Blogs:
>>> http://www.r-statistics.com/
>>> http://www.talgalili.com
>>> http://www.biostatistics.co.il
>>>
>>>       [[alternative HTML version deleted]]
>>>
>>>
>>>
>> Charles C. Berry                            (858) 534-2098
>>                                           Dept of Family/Preventive
>> Medicine
>> E mailto:cberry at tajo.ucsd.edu               UC San Diego
>> http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego  
>> 92093-0901
>>
>>
>
>
> -- 
> ----------------------------------------------
>
>
> My contact information:
> Tal Galili
> Phone number: 972-50-3373767
> FaceBook: Tal Galili
> My Blogs:
> http://www.r-statistics.com/
> http://www.talgalili.com
> http://www.biostatistics.co.il
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius, MD
Heritage Laboratories
West Hartford, CT




More information about the R-help mailing list