[R] Non-parametric test for location with two unpaired sets of data measured on ordinal scale.

[Peter Ehlers]

Greg and Marc,

Not that it's needed here but, of course, perm.test() in
pkg:exactRankTests or oneway_test() in pkg:coin can be
used.

Using Marc's / Greg's computations, the (two-sided) p-value
is

  sum(abs(perms) >= abs(orig)) / length(perms)
  [1] 0.01937395

perm.test() and oneway_test give a p-value of 0.003249691,
using the "exact" option.

Question:
Why the difference?

Answer:
perm.test() uses the *mean* difference instead of the
median difference. (Easy to check: just replace 'median'
with 'mean' in Marc's computation of perms.)

As Greg correctly points out, different test statistics
can sensibly be used. But which statistic, mean difference
or median difference, is more appropriate for the given data?

Assumptions:

1. the null hypothesis is that the two sets of observations
    represent random samples from the same distribution;

2. the range of the distribution consists of a small set
    of integers.

Fire away!

Peter Ehlers


Greg Snow wrote:
> Thanks Marc,
> 
> The sampling is so easy that I often forget that we can do the exact permutation test for smaller samples (and I can never remember when small is small enough for this).  With the exact permutations we really don't need to do the prop.test or binom.test, I usually do that to get the confidence interval on the p-value due to sampling from the permutations rather than doing all possible (and this tells me if I need to increase the number of permutations to be sure my p-value is precise enough).  With all possible permutations, there is no sampling, and no need for an interval, the p-value is exact.
> 
> Thanks again, I need to remember combn.
>