[R] detection of outliers

[Christian Hennig]

On Thu, 23 Sep 2004 Phguardiol at aol.com wrote:

> Hi,
> this is both a statistical and a R question...
> what would the best way / test to detect an outlier value among a series of 10 to 30 values ? for instance if we have the following dataset: 10,11,12,15,20,22,25,30,500 I d like to have a way to identify the last data as an outlier (only one direction). One way would be to calculate abs(mean - median) and if elevated (to what extent ?) delete the extreme data then redo.. but is it valid to do so with so few data ? is the (trimmed mean - mean) more efficient ? if so, what would be the maximal tolerable value to use as a threshold ? (I guess it will be experiment dependent...) tests for skweness will probably required a larger dataset ? 
> any suggestions are very welcome !
> thanks for your help
> Philippe Guardiola, MD
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> 

You may want to read 
Davies and Gather, The identification of multiple outliers, JASA 88 (1993),
782-801.

The simplest recommendation is to nominate all points with distance larger
than c*mad(data) from the median as outliers. Choices of c depending on n
are given in the above paper.

This is somewhat better founded theoretically than the boxplot method
recommended by Gabor G., but it is based on the assumption that the
distribution on the non-outliers is close to the normal and especially not
strongly skewed (the boxplot method
seems to be a bit more robust against skewness).

Christian
 
***********************************************************************
Christian Hennig
Fachbereich Mathematik-SPST/ZMS, Universitaet Hamburg
hennig at math.uni-hamburg.de, http://www.math.uni-hamburg.de/home/hennig/
#######################################################################
ich empfehle www.boag-online.de