[R] Can I use "mcnemar.test" for 3*3 tables (or is there a bug in the command?)

[peter dalgaard]

> On 30 Aug 2020, at 20:11 , Kyungmin Ko <kaymin.ko using gmail.com> wrote:
> 
>> 2) The second one is to produce a 3 by 3 table, with the rows indicating
>> what the kids answered to setting 1 of the experiment, and the columns
>> indicating the kids answers to setting 2.
>> Now the question is:
>> was there marginal homogenity? if not, then that is an indicator that the
>> general response to the experimental settings was different for the kids.
> 
>> 1) can I use "mcnemar.test" for 3*3 (or more) tables ?
> As Peter Dalgaard and jchavez12 (on Nabble) pointed out,

Um, that was on July 19, 2009.....

> "mcnemar.test" does McNemar-Bowker test (Bowker's test) which tests
> symmetry, not marginal homogeneity.
> Marginal homogeneity and symmetry are equivalent in 2x2 matrices but
> not for larger tables.
> I also was confused because many resources (for example Wikipedia
> accessed 2020-08-30) introduce McNemar's test as a test of marginal
> homogeneity (which it is only for 2x2 matrices).
> The R Reference Manual entry for mcnemar.test states that it tests for symmetry.
> The code for mcnemar.test is also consistent with the McNemar-Bowker
> test for symmetry.
> 
>> Is there a bug in the command?
> Since the function does what the manual states it does, I would not
> call this a bug.
> Although, I would like it if the result of mcnemar.test would print
> "McNemar's Chi-squared test of symmetry" rather than just "McNemar's
> Chi-squared test."
> 
>> Is the one necessarily better then the other? (for example for
>> sparser matrices ?)
> mcnemar.test often fails for sparse matrices, because symmetric zeros
> produce a NaN due to division by zero in the following line of
> mcnemar.test code:
> STATISTIC <- sum(y[upper.tri(x)]^2/x[upper.tri(x)])
> The McNemar-Bowker test uses Chi-squared approximation, which would
> not be good for small counts (and sparse matrices).
> mcnemar.test does not perform continuity correction for matrices
> larger than 2x2.
> Is there an exact test for symmetry of matrices larger than 2x2?
> I could not find one.

The exact test for the 2x2 case is isomorphic to a binomial test of the two off-diagonal elements (conditioning on the sum).

The natural way of constructing a test for the k x k case would be based on the Cartesian product of k(k-1)/2 binomials with p=.5, (one for each i,j-combination). This shouldn't be too hard if k is small, but of course it explodes combinatorially as k increases.

-pd

> 
>> Is there a bug in the command?
> I would not call this a bug.
> This behavior of giving "NA" due to a division by zero seems to be
> consistent across statistical tests in R.
> For example chisq.test(matrix(c(0, 0, 1, 2), nrow = 2) gives NA.
> 
>> So which one is "right" ?
> You have the option of the McNemar-Bowker test for symmetry
> (mcnemar.test), and Stuart-Maxwell test (mh_test).
> As an "indicator that the general response to the experimental
> settings was different for the kids,"
> I would think that if marginal homogeneity is rejected, the two tests
> are not equivalent.
> I would run mh_test with distribution = "exact" .
> The relationship between symmetry and equivalence of two tests is not
> as clear to me.
> I suppose if the two experimental settings are equivalent and the
> distribution of random error for each test are also the same the
> resulting matrix would be symmetric?
> 
> R 4.0.2 . coin 1.3.1 .
> 
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd.mes using cbs.dk  Priv: PDalgd using gmail.com