[R-sig-ME] same old question - lme4 and p-values

[Nicholas Lewin-Koh]

Hi,
I feel compelled to ad my 2c here. I am strongly against
the reporting of p-values for several reasons. There are the slew
of theoretical arguments about p-values, the most compelling is which is
the
correct null distribution, and that p-values are a very measure of
statistical 
evidence, if they measure evidence at all. p-values only measure the
null versus
alternative, and are not a measure of evidence of one model over another
that
can be ranked over a set of alternate models. See Royall 1997, or Taper
and Lele 2004.

These arguments aside, in the world where I work, early development
pharmaceuticals, many scientists and
managers would have the p-values make decisions for them. This is very
dangerous, as the
experiment to experiment variation is usually much higher than the
within experiment variation.
Even more important, it is not clear to me that many of the processes we
look
at converge to a stable distribution. I think that the responsibility of
the scientist 
in a publication is to show the variability of the data, explain the
sources of 
variation and how they were controlled, the relevant effect sizes and
the ASSUMPTIONS
made in the model. In either the epidemiological or the experimental
paradigms
consensus will only come with demonstrated repeatability. A publication
should
generate scientific discussion about the mechanisms and trends being
reported. I am
not sure that p-values generate the right discussion.

Didn't mean to rant.

Nicholas

Statistical Evidence: A Likelihood Paradigm. R. Royall, Chapman & Hall,
London, 1997

MARK L. TAPER AND SUBHASH R. LELE, eds. The Nature of Scientific
Evidence: Statistical, Philosophical, 
and Empirical Considerations. Chicago and London: University of Chicago
Press, 2004.