On Sat, May 8, 2010 at 7:02 PM, Bak Kuss <bakkuss@gmail.com> wrote:

>
> Just wondering.
>
> The smallest the p-value, the closer  to 'reality'  (the more accurate)
> the model is supposed to (not) be (?).
>
> How realistic is it to be that (un-) real?
>

That's a common misconception. A p-value expresses no more than the chance
of obtaining the dataset you observe, given that your null hypothesis _and
your assumptions_ are true. Essentially, a p-value is as "real" as your
assumptions. In that way I can understand what Robert wants to say. But with
lare enough datasets, bootstrapping or permutation tests gives often about
the same p-value as the asymptotic approximation. At that moment, the
central limit theorem comes into play, which says that when the sample size
is big enough, the mean is -close to- normally distributed. In those cases,
the test statistic also follows the proposed distribution and your p-value
is closer to "reality". Mind you, the "sample size" for a specific statistic
is not always merely the number of observations, especially in more advanced
methods. Plus, violations of other assumptions, like independence of the
observations, changes the picture again.

The point is : what is reality? As Duncan said, a small p-value indicates
that your null hypothesis is not true. That's exactly what you look for,
because that is the proof the relation in your dataset you're looking at,
did not emerge merely by chance. You're not out to calculate the exact
chance. Robert is right, reporting an exact p-value of 1.23 e-7 doesn't make
sense at all. But the rejection of your null-hypothesis is as real as life.

The trick is to test the correct null hypothesis, and that's were it most
often goes wrong...

Cheers
Joris

> bak
>
> p.s. I am no statistician
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Joris Meys
Statistical Consultant

Ghent University
Faculty of Bioscience Engineering
Department of Applied mathematics, biometrics and process control

Coupure Links 653
B-9000 Gent

tel : +32 9 264 59 87
Joris.Meys@Ugent.be
-------------------------------
Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php

	[[alternative HTML version deleted]]

