Estimating the Proportion of False Null Hypotheses among a Large Number of Independently Tested Hypotheses

Nicolai Meinshausen and John Rice

Oktober 2004

Abstract:
We consider the problem of estimating the number of false null hypotheses among a very large number of independently tested
hypotheses, focusing on the situation in which the proportion of false null hypotheses is very small. We propose a family of
methods for establishing lower 100(1-alpha)\% confidence bounds for this proportion, based on the empirical distribution of the
p-values of the tests. Methods in this family are then compared in terms of ability to consistently estimate the proportion by
letting alpha get to 0 as the number of hypothesis tests increases and the proportion decreases. This work is motivated by a
signal detection problem occurring in astronomy.

Download: Compressed Postscript (130 Kb) PDF (335 Kb).