[R] Binom.test - hudge difference in p-value for little differences in PD forecast

[peter dalgaard]

On 10 Apr 2014, at 16:49 , Alwina Hermann <alwina.hermann at 1plusi.de> wrote:

> Dear R team,
>  
> I'm not sure if I use the right distribution list, but I hope in case if
> not, you will forward it to the reference person.
>  
> Following problem occured:
> I used R to calculate the p-value for the two sided binomial test (exact -
> Pearson).
> For a very little difference for my forecast I get a very big difference in
> my p-value
>  
>  >  binom.test(1,101, 0.02402)
> 
>         Exact binomial test
> 
> data:  1 and 101
> number of successes = 1, number of trials = 101, p-value = 0.7375
> alternative hypothesis: true probability of success is not equal to 0.02402
> 95 percent confidence interval:
> 0.00025064 0.05393235
> sample estimates:
> probability of success 
>             0.00990099
> 
>> binom.test(1,101, 0.02403)
> 
>         Exact binomial test
> 
> data:  1 and 101
> number of successes = 1, number of trials = 101, p-value = 0.5243
> alternative hypothesis: true probability of success is not equal to 0.02403
> 95 percent confidence interval:
> 0.00025064 0.05393235
> sample estimates:
> probability of success 
>             0.00990099
> 
> 
> Can you please explain where this huge difference come from?
> Which mathematical explanation is given for this topic?

Check the definition of the p value (you need to study the source for that, I suppose). It's the probability of getting an observation with a point probability less than or equal to that of the observed value. The crucial bit is whether the point probability p(1) is less than or bigger than p(3):

> dbinom(0:4,101, 0.02403)
[1] 0.08572022 0.21316798 0.26242746 0.21322618 0.12862456
> dbinom(0:4,101, 0.02402)
[1] 0.08580898 0.21329771 0.26247520 0.21317403 0.12853828

I.e., in the first case, X==3 is not counted into the p-value, whereas it is in the second case.


-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com