[R] null distribution of binom.test p values

[Chris Wallace]

Greg, thanks for the reply.

Unfortunately, I remain unconvinced!

I ran a longer simulation, 100,000 reps.  The size of the test is 
consistently too small (see below) and the histogram shows increasing 
bars even within the parts of the histogram with even bar spacing.  See 
https://www-gene.cimr.cam.ac.uk/staff/wallace/hist.png

y<-sapply(1:100000, function(i,n=100)
  binom.test(sum(rnorm(n)>0),n,p=0.5,alternative="two")$p.value)
mean(y<0.01)
# [1] 0.00584
mean(y<0.05)
# [1] 0.03431
mean(y<0.1)
# [1] 0.08646

Can that really be due to the discreteness of the distribution?

C.

On 26/01/12 16:08, Greg Snow wrote:
> I believe that what you are seeing is due to the discrete nature of the binomial test.  When I run your code below I see the bar between 0.9 and 1.0 is about twice as tall as the bar between 0.0 and 0.1, but the bar between 0.8 and 0.9 is not there (height 0), if you average the top 2 bars (0.8-0.9 and 0.9-1.0) then the average height is similar to that of the lowest bar.  The bar between 0.5 and 0.6 is also 0, if you average that one with the next 2 (0.6-0.7 and 0.7-0.8) then they are also similar to the bars near 0.
>
>
>