[R] normality testing with nortest

[Peter Dalgaard]

Uwe Ligges <ligges at statistik.uni-dortmund.de> writes:


> > A = 0.1846, p-value = 0.9059
> > 
> >> ad.test(rnorm(100,mean=5,sd=3))
> > 
> > ...
> > A = 0.5138, p-value = 0.1887
> > 
> >     I mistakenly had thought the p-values would be more stable since I 
> > am artificially creating a random normal distribution.  Is this expected 
> > for a normality test or is this an issue with how rnorm is producing 
> > random numbers?  I guess if I run it many times, I would find that I 
> > would get many large values for the p-value?
> 
> Well, as many large values as small values, 5% significant differences 
> for the 5% level....
> 
> The following looks alright:
> 
> hist(replicate(1000,  ad.test(rnorm(100,mean=5,sd=3))$p.value))

We see this misunderstanding worryingly often. Worrying because it
reveals that a fundamental aspect of statistical inference has not
been grasped: that p-values are designed to be (approximately)
uniformly distributed and fall below any given level with the stated
probability, when the null hypothesis is true.

There is no mechanism to give you "fewer significant" or "more stable"
p-values, and a p-value close to one is no better an indication of a
true null hypothesis than one of 0.5 or 0.25.

-- 
   O__  ---- Peter Dalgaard             Øster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX: (+45) 35327907