[R] Testing for normality of residuals in a regression model

[Prof Brian Ripley]

I am assuming everyone is on R-help and doesn't want two copies so have 
trimmed the Cc: list to R-help.

On Sat, 16 Oct 2004, Philippe Grosjean wrote:

> > Prof Brian Ripley wrote:

[ Other contributions previously excised here without comment. ]

> > >>However, stats 901 or some such tells you that if the distributions 
> > >>have even slightly longer tails than the normal you can get much 
> > >>better estimates than OLS, and this happens even before a test of 
> > >>normality rejects on a sample size of thousands.
> > >>
> > >>Robustness of efficiency is much more important than robustness of 
> > >>distribution, and I believe robustness concepts should be 
> > in stats 101.
> > >>(I was teaching them yesterday in the third lecture of a  basic course, 
> > >>albeit a graduate course.)
> 
> Federico Gherardini answered:
> > This is a very interesting discussion. So you are basically 
> > saying that it's better to use robust regression methods, 
> > without having to worry too much about the distribution of 
> > residuals, instead of using standard methods and doing a lot 
> > of tests to check for normality? Did I get your point?
> 
> My feeling is that symmetry is more important than, let's say kurtosis <> 0
> in the error. Is this correct? Now the problem is: the lower number of
> observations, the more severe an effect of non-normality (at least,
> asymmetry?) could be on the regression AND at the same time, power of tests
> to detect non normality drops. So, I can imagine easily situations where
> non-normality is not detected, yet asymmetry is such that regression is
> significantly biased... 

Before you can even talk about bias you have to agree what it is you are
trying to estimate.  For asymmetric error distributions it is unlikely to
be the population mean, but if it is then least-squares linear regression
is unbiased provided only that the error distribution has a finite first
moment.  (Part of the so-called Gauss-Markov Theorem.  This seems to
suggest that Philippe's `easy imagination' is of impossible things.)

For contaminated normal distributions it is possibly the mean of the
uncontaminated normal component, and the latter seems the commonest aim of
mainstream robust methods, which do often assume symmetry.  (This may not
affect interpretation of coefficients other than the intercept.)  The
(non-linear) robust regression estimators may be biased for the population
mean but have a (much) smaller variability for long-tailed distributions.

There is a lot of careful discussion about this in the statistical
literature, and I don't believe that it is profitable for people to be
discussing this without knowing the literature, and probably not _here_
even then.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595