[R] [EXT] A very small p-value

[Christophe Dutang]

Thanks all for your answers.

It seems most of my students understand that the way to compute a small probability may impact the result. 

Regards, CD

> Le 4 nov. 2025 à 20:43, Eik Vettorazzi <E.Vettorazzi using uke.de> a écrit :
> 
> Hi,
> Stepping briefly outside the R context, I noticed a statistical point in the text you linked that, in my opinion, isn't quite right. I believe there's a key misunderstanding here: The statement that the z-test does not depend on the number of cases is incorrect. The p-value of the z-test is —just like other tests— very much dependent on the sample size, assuming the same mean difference and standard deviation.
> The text you linked is actually calculating an Effect Size, which is (largely) independent of the sample size. Effect Size answers the question of how "relevant" or "large" the difference between groups is. This is fundamentally different from testing for "significant" differences.
> Specifically, the crucial 1/\sqrt{n} term, which is necessary for calculating the standard error of the mean difference, seems to be missing from the presented formula for the z-score. I just wanted to quickly point this out.
> 
> Best regards
> 
> Am 27.10.2025 um 14:12 schrieb Petr Pikal:
>> Hallo
>> The t test is probably not the best option in your case. With 95
>> observations your data behave more like a population and you  may get
>> better insight using z-test. See
>> https://toxictruthblog.com/avoiding-little-known-problems-with-the-t-test/
>> Best regards.
>> Petr
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>> Neobsahuje
>> žádné viry.www.avast.com
>> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
>> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>> so 25. 10. 2025 v 11:46 odesílatel Christophe Dutang <dutangc using gmail.com>
>> napsal:
>>> Dear list,
>>> 
>>> I'm computing a p-value for the Student test and discover some
>>> inconsistencies with the cdf pt().
>>> 
>>> The observed statistic is 11.23995 for 95 observations, so the p-value is
>>> very small
>>> 
>>>> t_score <- 11.23995
>>>> n <- 95
>>>> print(pt(t_score, df = n-2, lower=FALSE), digits=22)
>>> [1] 2.539746620181247991746e-19
>>>> print(integrate(dt, lower=t_score, upper=Inf, df=n-2)$value, digits = 22)
>>> [1] 2.539746631161970791961e-19
>>> 
>>> But if I compute with pt(lower=TRUE), I got 0
>>> 
>>>> print(1-pt(t_score, df = n-2, lower=TRUE), digits=22)
>>> [1] 0
>>> 
>>> Indeed, the p-value is lower than the epsilon machine
>>> 
>>>> pt(t_score, df = n-2, lower=FALSE) < .Machine$double.eps
>>> [1] TRUE
>>> 
>>> Using the square of t statistic which follows a Fisher distribution, I got
>>> the same issue:
>>> 
>>>> print(pf(z, 1, n-2, lower=FALSE), digits=22)
>>> [1] 5.079493240362495983491e-19
>>>> print(integrate(df, lower=z, upper=Inf, df1=1, df2=n-2)$value, digits =
>>> 22)
>>> [1] 5.079015231299358486828e-19
>>>> print(1-pf(z, 1, n-2, lower=TRUE), digits=22)
>>> [1] 0
>>> 
>>> When using the t.test() function, the p-value is naturally printed :
>>> p-value < 2.2e-16.
>>> 
>>> Any comment is welcome.
>>> 
>>> Christophe
>>> 
>>>> R.version
>>>                _
>>> platform       aarch64-apple-darwin20
>>> arch           aarch64
>>> os             darwin20
>>> system         aarch64, darwin20
>>> status
>>> major          4
>>> minor          5.1
>>> year           2025
>>> month          06
>>> day            13
>>> svn rev        88306
>>> language       R
>>> version.string R version 4.5.1 (2025-06-13)
>>> nickname       Great Square Root
>>> -------------------------------------------------
>>> Christophe DUTANG
>>> LJK, Ensimag, Grenoble INP, UGA, France
>>> ILB research fellow
>>> Web: http://dutangc.free.fr
>>> -------------------------------------------------
>>> 
>>> 
>>>         [[alternative HTML version deleted]]
>>> 
>>> ______________________________________________
>>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> https://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>> 
>> 	[[alternative HTML version deleted]]
>> ______________________________________________
>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> 
> -- 
> Eik Vettorazzi
> 
> Universitätsklinikum Hamburg-Eppendorf
> Institut für Medizinische Biometrie und Epidemiologie
> 
> Christoph-Probst-Weg 1
> 4. Obergeschoss, Raum 04.1.021.1
> 
> 20246 Hamburg
> 
> Telefon: +49 (0) 40 7410 - 58243
> Fax:     +49 (0) 40 7410 - 57790
> 
> Web: www.uke.de/imbe
> 
> 
> 
> --
> 
> _____________________________________________________________________
> 
> Universitätsklinikum Hamburg-Eppendorf; Körperschaft des öffentlichen Rechts; Gerichtsstand: Hamburg | www.uke.de
> Vorstandsmitglieder: Prof. Dr. Christian Gerloff (Vorsitzender), Joachim Prölß, Prof. Dr. Blanche Schwappach-Pignataro, Corinna Wriedt
> _____________________________________________________________________