[R] general question on binomial test / sign test

Thu Sep 2 10:42:38 CEST 2010

You state: "in reverse the p-value of 1 says that i can 100% sure
that the estimate of 0.5 is true". This is where your logic about
significance tests goes wrong.

The general logic of a singificance test is that a test statistic
(say T) is chosen such that large values represent a discrepancy
between possible data and the hypothesis under test. When you
have the data, T evaluates to a value (say t0). The null hypothesis
(NH) implies a distribution for the statistic T if the NH is true.

Then the value of Prob(T >= t0 | NH) can be calculated. If this is
small, then the probability of obtaining data at least as discrepant
as the data you did obtain is small; if sufficiently small, then
the conjunction of NH and your data (as assessed by the statistic T)
is so unlikely that you can decide to not believe that it is possible.
If you so decide, then you reject the NH because the data are so
discrepant that you can't believe it!

This is on the same lines as the "reductio ad absurdum" in classical
logic: "An hypothesis A implies that an outcome B must occur. But I
have observed that B did not occur. Therefore A cannot be true."

But it does not follow that, if you observe that B did occur
(which is *consistent* with A), then A must be true. A could be
false, yet B still occur -- the only basis on which occurrence
of B could *prove* that A must be true is when you have the prior
information that B will occur *if and only if* A is true. In the
reductio ad absurdum, and in the parallel logic of significance
testing, all you have is "B will occur *if* A is true". The "only if"
part is not there. So you cannot deduce that "A is true" from
the observation that "B occurred", since what you have to start with
allows B to occur if A is false (i.e. "B will occur *if* A is true"
says nothing about what may or may not happen if A is false).

So, in your single toss of a coin, it is true that "I will observe
either 'succ' or 'fail' if the coin is fair". But (as in the above)
you cannot deduce that "the coin is fair" if you observe either
'succ' or 'fail', since it is possible (indeed necessary) that you
obtain such an observation if the coin is not fair (even if the
coin is the same, either 'succ' or 'fail', on both sides, therefore
completely unfair). This is an attempt to expand Greg Snow's reply!

Your 2-sided test takes the form T=1 if either outcome='succ' or
outcome='fail'. And that is the only possible value for T since
no other outcome is possible. Hence Prob(T==1) = 1 whether the coin
is fair or not. It is not possible for such data to discriminate
between a fair and an unfair coin.

And, as explained above, a P-value of 1 cannot prove that the
null hypothesis is true. All that is possible with a significance
test is that a small P-value can be taken as evidence that the
NH is false.

Hoping this helps!
Ted.

On 02-Sep-10 07:41:17, Kay Cecil Cichini wrote:
> i test the null that the coin is fair (p(succ) = p(fail) = 0.5) with  
> one trail and get a p-value of 1. actually i want to proof the  
> alternative H that the estimate is different from 0.5, what certainly  
> can not be aproven here. but in reverse the p-value of 1 says that i  
> can 100% sure that the estimate of 0.5 is true (??) - that's the point 
> that astonishes me.
> 
> thanks if anybody could clarify this for me,
> kay
> 
> Zitat von Greg Snow <Greg.Snow at imail.org>:
> 
>> Try thinking this one through from first principles, you are  
>> essentially saying that your null hypothesis is that you are  
>> flipping a fair coin and you want to do a 2-tailed test.  You then  
>> flip the coin exactly once, what do you expect to happen?  The  
>> p-value of 1 just means that what you saw was perfectly consistent  
>> with what is predicted to happen flipping a single time.
>>
>> Does that help?
>>
>> If not, please explain what you mean a little better.
>>
>> --
>> Gregory (Greg) L. Snow Ph.D.
>> Statistical Data Center
>> Intermountain Healthcare
>> greg.snow at imail.org
>> 801.408.8111
>>
>>
>>> -----Original Message-----
>>> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-
>>> project.org] On Behalf Of Kay Cichini
>>> Sent: Wednesday, September 01, 2010 3:06 PM
>>> To: r-help at r-project.org
>>> Subject: [R] general question on binomial test / sign test
>>>
>>>
>>> hello,
>>>
>>> i did several binomial tests and noticed for one sparse dataset that
>>> binom.test(1,1,0.5) gives a p-value of 1 for the null, what i can't
>>> quite
>>> grasp. that would say that the a prob of 1/2 has p-value of 0 ?? - i
>>> must be
>>> wrong but can't figure out the right interpretation..
>>>
>>> best,
>>> kay
>>>
>>>
>>>
>>>
>>>
>>> -----
>>> ------------------------
>>> Kay Cichini
>>> Postgraduate student
>>> Institute of Botany
>>> Univ. of Innsbruck
>>> ------------------------
>>>
>>> --
>>> View this message in context: http://r.789695.n4.nabble.com/general-
>>> question-on-binomial-test-sign-test-tp2419965p2419965.html
>>> Sent from the R help mailing list archive at Nabble.com.
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-
>>> guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at manchester.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 02-Sep-10                                       Time: 09:42:34
------------------------------ XFMail ------------------------------