# [R] about a p-value < 2.2e-16

Jiefei Wang @zwj|08 @end|ng |rom gm@||@com
Fri Mar 19 15:01:51 CET 2021

```Hey,

I just want to point out that the word "exact" has two meanings. It can
mean the numerically accurate p-value as Bogdan asked in his first email,
or it could mean the p-value calculated from the exact distribution of the
statistic(In this case, U stat). These two are actually not related, even
though they all called "exact".

Best,
Jiefei

On Fri, Mar 19, 2021 at 9:31 PM Spencer Graves <
spencer.graves using effectivedefense.org> wrote:

>
>
> On 2021-3-19 12:54 AM, Bogdan Tanasa wrote:
> > thanks a lot, Vivek ! in other words, assuming that we work with 1000
> data
> > points,
> >
> > shall we use EXACT = TRUE, it uses the normal approximation,
> >
> > while if EXACT=FALSE (for these large samples), it does not ?
>
>
>        As David Winsemius noted, the documentation is not clear.
> Consider the following:
>
> > set.seed(1)  > x <- rnorm(100) > y <- rnorm(100, 2) > > wilcox.test(x,
> y)\$p.value
> [1] 1.172189e-25 > wilcox.test(x, y)\$p.value [1] 1.172189e-25 > >
> wilcox.test(x, y, EXACT=TRUE)\$p.value [1] 1.172189e-25 > wilcox.test(x,
> y, EXACT=TRUE)\$p.value [1] 1.172189e-25 > wilcox.test(x, y,
> exact=TRUE)\$p.value [1] 4.123875e-32 > wilcox.test(x, y,
> exact=TRUE)\$p.value [1] 4.123875e-32 > > wilcox.test(x, y,
> EXACT=FALSE)\$p.value [1] 1.172189e-25 > wilcox.test(x, y,
> EXACT=FALSE)\$p.value [1] 1.172189e-25 > wilcox.test(x, y,
> exact=FALSE)\$p.value [1] 1.172189e-25 > wilcox.test(x, y,
> exact=FALSE)\$p.value [1] 1.172189e-25 > We get two values here:
> 1.172189e-25 and 4.123875e-32. The first one, I think, is the normal
> approximation, which is the same as exact=FALSE. I think that with
> exact=FALSE, you get a permutation distribution, though I'm not sure.
> You might try looking at "wilcox_test in package coin for exact,
> asymptotic and Monte Carlo conditional p-values, including in the
> presence of ties" to see if it is clearer. NOTE: R is case sensitive, so
> "EXACT" is a different variable from "exact". It is interpreted as an
> optional argument, which is not recognized and therefore ignored in this
> context.
>           Hope this helps.
>           Spencer
>
>
> > On Thu, Mar 18, 2021 at 10:47 PM Vivek Das <vd4mmind using gmail.com> wrote:
> >
> >> Hi Bogdan,
> >>
> >> You can also get the information from the link of the Wilcox.test
> function
> >> page.
> >>
> >> “By default (if exact is not specified), an exact p-value is computed if
> >> the samples contain less than 50 finite values and there are no ties.
> >> Otherwise, a normal approximation is used.”
> >>
> >> For more:
> >>
> >>
> https://stat.ethz.ch/R-manual/R-devel/library/stats/html/wilcox.test.html
> >>
> >> Hope this helps!
> >>
> >> Best,
> >>
> >> VD
> >>
> >>
> >> On Thu, Mar 18, 2021 at 10:36 PM Bogdan Tanasa <tanasa using gmail.com>
> wrote:
> >>
> >>> Dear Peter, thanks a lot. yes, we can see a very precise p-value, and
> that
> >>> was the request from the journal.
> >>>
> >>> if I may ask another question please : what is the meaning of
> "exact=TRUE"
> >>> or "exact=FALSE" in wilcox.test ?
> >>>
> >>> i can see that the "numerically precise" p-values are different.
> thanks a
> >>> lot !
> >>>
> >>> tst = wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE)
> >>> tst\$p.value
> >>> [1] 8.535524e-25
> >>>
> >>> tst = wilcox.test(rnorm(100), rnorm(100, 2), exact=FALSE)
> >>> tst\$p.value
> >>> [1] 3.448211e-25
> >>>
> >>> On Thu, Mar 18, 2021 at 10:15 PM Peter Langfelder <
> >>> peter.langfelder using gmail.com> wrote:
> >>>
> >>>> I thinnk the answer is much simpler. The print method for hypothesis
> >>>> tests (class htest) truncates the p-values. In the above example,
> >>>>
> >>>> wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE)
> >>>>
> >>>> and copying the output, just print the p-value:
> >>>>
> >>>> tst = wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE)
> >>>> tst\$p.value
> >>>>
> >>>> [1] 2.988368e-32
> >>>>
> >>>>
> >>>> I think this value is what the journal asks for.
> >>>>
> >>>> HTH,
> >>>>
> >>>> Peter
> >>>>
> >>>> On Thu, Mar 18, 2021 at 10:05 PM Spencer Graves
> >>>> <spencer.graves using effectivedefense.org> wrote:
> >>>>>         I would push back on that from two perspectives:
> >>>>>
> >>>>>
> >>>>>               1.  I would study exactly what the journal said very
> >>>>> carefully.  If they mandated "wilcox.test", that function has an
> >>>>> argument called "exact".  If that's what they are asking, then using
> >>>>> that argument gives the exact p-value, e.g.:
> >>>>>
> >>>>>
> >>>>>   > wilcox.test(rnorm(100), rnorm(100, 2), exact=TRUE)
> >>>>>
> >>>>>           Wilcoxon rank sum exact test
> >>>>>
> >>>>> data:  rnorm(100) and rnorm(100, 2)
> >>>>> W = 691, p-value < 2.2e-16
> >>>>>
> >>>>>
> >>>>>               2.  If that's NOT what they are asking, then I'm not
> >>>>> convinced what they are asking makes sense:  There is is no such
> thing
> >>>>> as an "exact p value" except to the extent that certain assumptions
> >>>>> hold, and all models are wrong (but some are useful), as George Box
> >>>>> famously said years ago.[1]  Truth only exists in mathematics, and
> >>>>>
> >>>>>
> >>>>>         Hope this helps.
> >>>>>         Spencer Graves
> >>>>>
> >>>>>
> >>>>> [1]
> >>>>> https://en.wikipedia.org/wiki/All_models_are_wrong
> >>>>>
> >>>>>
> >>>>> On 2021-3-18 11:12 PM, Bogdan Tanasa wrote:
> >>>>>>    <
> >>>>
> >>>>
> >>>>>> Dear all,
> >>>>>>
> >>>>>>
> >>>>>> in R, the wilcox.test() provides "a p-value < 2.2e-16", when we
> >>> compare
> >>>>>> sets of 1000 genes expression (in the genomics field).
> >>>>>>
> >>>>>> however, the journal asks us to provide the exact p value ...
> >>>>>>
> >>>>>> would it be legitimate to write : "p-value = 0" ? thanks a lot,
> >>>>>>
> >>>>>> -- bogdan
> >>>>>>
> >>>>>>        [[alternative HTML version deleted]]
> >>>>>>
> >>>>>> ______________________________________________
> >>>>>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>>> http://www.R-project.org/posting-guide.html
> >>>>>> and provide commented, minimal, self-contained, reproducible code.
> >>>>> ______________________________________________
> >>>>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >>>>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>>> http://www.R-project.org/posting-guide.html
> >>>>> and provide commented, minimal, self-contained, reproducible code.
> >>>          [[alternative HTML version deleted]]
> >>>
> >>> ______________________________________________
> >>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>> http://www.R-project.org/posting-guide.html
> >>> and provide commented, minimal, self-contained, reproducible code.
> >>>
> >> --
> >> ----------------------------------------------------------
> >>
> >> Vivek Das, PhD
> >>
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help