[R] binom.test & p-value

[peter dalgaard]

No, there is NO lower critical value at 0.05 level since already P(X<=0) > .05. 

The basic issue here (generic to all two-sided 1 dimensional "exact" tests) is how to compute the rejection region. 

The version used by R is to count into the total p-value all points with a lower point probability than the one observed. So for X=6, we do

 > dbinom(0:26,26,.1)
 [1] 6.461082e-02 1.866535e-01 2.592409e-01 2.304364e-01 1.472232e-01
 [6] 7.197581e-02 2.799059e-02 8.885903e-03 2.344891e-03 5.210869e-04
[11] 9.842752e-05 1.590748e-05 2.209372e-06 2.643693e-07 2.727620e-08
[16] 2.424551e-09 1.852087e-10 1.210515e-11 6.725082e-13 3.146237e-14
[21] 1.223537e-15 3.884243e-17 9.808695e-19 1.895400e-20 2.632500e-22
[26] 2.340000e-24 1.000000e-26
> dbin <- dbinom(0:26,26,.1)
> dbin[7]
[1] 0.02799059
> sum(dbin[7:27])
[1] 0.03985931
> dbin < dbin[7]
 [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE
[13]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[25]  TRUE  TRUE  TRUE
> sum(dbin[dbin<=dbin[7]])
[1] 0.03985931

Notice that all points with lower dbin are to the right of X==6. For X==5 we get

> dbin < dbin[6]
 [1]  TRUE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[13]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[25]  TRUE  TRUE  TRUE
> sum(dbin[dbin<=dbin[6]])
[1] 0.1764459

so now we do count the point mass at X==0, but the total p-value is not < .05

Similarly, we do not reject if X==0
> sum(dbin[dbin<=dbin[1]])
[1] 0.1044701

where as we obviously rejece for any X>6. So we end up with a one-sided rejection region for any alpha value < .1045.

This is not uncontroversial, some may want to construct the test as two 1-sided test at half the significance level. 
In that case, P(X>=6) > .025 and no rejection at .05 level.
 
This approach has the benefit of being consistent with confidence interval calculations, but it does not generalize to multidimensional cases like fisher.test() on a 3x2 table.

-pd

> On 4 Apr 2022, at 08:38 , Sigbert Klinke <sigbert using wiwi.hu-berlin.de> wrote:
> 
> Hi,
> 
> the lower critical value is zero since:
> 
> > cbind(0:2, pbinom(0:2, 26, 0.1))
> 
>     [,1]       [,2]
> 
> [1,]    0 0.06461082
> 
> [2,]    1 0.25126430
> 
> [3,]    2 0.51050524
> 
> the upper critical value c_u=6 since:
> 
> > cbind(5:7, pbinom((5:7), 26, 0.1))
> 
>     [,1]      [,2]
> 
> [1,]    5 0.9601407
> 
> [2,]    6 0.9881313
> 
> [3,]    7 0.9970172
> 
> with F(c_u)>=0.975 and F(c_u-1)<0.975 .
> 
> Thus,
> 
> * x=6 is in [0,6] => can not reject null
> * p-value 0.03985 < 0.05 => reject null
> 
> Am 03.04.22 um 20:19 schrieb Rui Barradas:
>> Hello,
>> Can you post your computations? Assuming a two-sided test, mine are
>> x <- 6
>> n <- 26
>> p <- 0.1
>> cmb <- sapply(x:n, \(i) choose(n, i))
>> sum(cmb * p^(x:n) * (1 - p)^(n - (x:n)))
>> #[1] 0.03985931
>> binom.test(x=6, n=26, p=0.1)$p.value
>> #[1] 0.03985931
>> The result are equal to one another.
>> Às 19:00 de 03/04/2022, Sigbert Klinke escreveu:
>>> Hi,
>>> 
>>> for the specific example binom.test(x=6, n=26, p=0.1) I get as p-value 0.03986. The default approach to decide whether I can reject the null or or not is to compare the p-value with the given significance level. Using a significance level of 0.05 this will lead to reject the null hypothesis.
>>> 
>>> However, computing things by hand it turned out that the critical values are 0 and 6. Since the test statistic is also 6 I can not reject the null hypothesis.
>>> 
>>> I found the discussion under https://stat.ethz.ch/pipermail/r-help/2009-February/380341.html and I understand that a p-value is not well defined if we have a asymmmetric (discrete) distribution under the null.
>>> 
>>> At least I would have expected some hint in the documentation for binom.test.
>>> 
>>> Sigbert
>>> 
> 
> 
> -- 
> https://hu.berlin/sk
> https://www.stat.de/faqs
> https://hu.berlin/mmstat
> https://hu.berlin/mmstat-int
> https://hu.berlin/mmstat-ar
> 
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd.mes using cbs.dk  Priv: PDalgd using gmail.com