[Rd] cor.test(method = spearman, exact = TRUE) not exact (PR#14095)

[dsimcha at gmail.com]

Full_Name: David Simcha
Version: 2.10
OS: Windows XP Home
Submission from: (NULL) (173.3.208.5)


> a <- c(1:10)
> b <- c(1:10)
> cor.test(a, b, method = "spearman", alternative = "greater", exact = TRUE)

        Spearman's rank correlation rho

data:  a and b 
S = 0, p-value < 2.2e-16
alternative hypothesis: true rho is greater than 0 
sample estimates:
rho 
  1 

> 1 / factorial(10)
[1] 2.755732e-07

Since we have perfect rank correlation and only one permutation out of 10! could
give this for N = 10, the p-value should be 1/10!.  Reading the code in prho.c,
it appears that the "exact" calculation uses the Edgeworth approximation for N >
9.  This makes sense because, for similar examples with N <= 9, the results are
as expected (1 / N!).

The "exact" p-value calculation is good enough for most practical purposes, but
is clearly not exact.  Some informal testing I've done indicates that it can
even be less accurate than the "approximate" p-value calculation in some cases. 
I think it's absurd to call these p-values "exact" when they are clearly based
on an asymptotic approximation that can be off by orders of magnitude in some
cases.