[BioC] adjustment for dependent hypotheses when pairwise testing

[Gordon K Smyth]

Actually I don't think the number of genes is a problem.  Neither BH nor BY use an estimate of pi0
(the true proportion of hull hypotheses).  Whenever pi0 would appear as a multiplier in the
formulae, they use 1 instead.  The methods are therefore somewhat conservative if pi0 is not
large, but they continue to control the FDR at below the requested level.

This means that BY gives rigorous control of expected FDR for any dependence structure and any
number of genes, given only that the p-value distribution is valid under the null hypothesis.  BH
is the same except that it is theoretically valid only for certain dependence structures (positive
regression dependence).

I think that Rebecca's question is about the particular (negative) dependence structure which is
found between the set of pairwise contrasts for a particular gene.  Yes, I think this does
theoretically invalidate BH.  However in practical situations you'll actually find the main
problem to be conserativism rather than the other way around because there are more pairwise
comparisons than there are independent comparisons to be made.  So, personally, this wouldn't make
me move to BY.

Hope this helps
Gordon

> Date: Fri, 13 Jan 2006 16:57:28 -0500
> From: Naomi Altman <naomi at stat.psu.edu>
> Subject: Re: [BioC] adjustment for dependent hypotheses when pairwise
> 	testing
> To: pmt1rew at leeds.ac.uk,	"bioconductor at stat.math.ethz.ch"
> 	<bioconductor at stat.math.ethz.ch>
>
> I do not think you will be able to use these methods with such a
> small number of genes.  A large number of genes are required to estimate pi0.
>
> --Naomi
>
> At 07:52 AM 1/13/2006, pmt1rew at leeds.ac.uk wrote:
>>Dear all,
>>
>>I data for 13 genes with the samples from 4 different tissue
>>types.  Would like
>>to compare the normal means of the different tissue types using pairwise
>>testing with an adjustment for fdr.
>>
>>Having considered the Benjamini and Yekutieli reference on the p.adjust help
>>page I am unsure as to whether my dependency structure satisfies the criteria
>>to just use the regular "BH" adjustment or if  the "BY" adjustment would be
>>more appropriate?
>>
>>Any suggestions for me please?
>>
>>Many thanks
>>
>>Rebecca Walls
>>
> Naomi S. Altman                                814-865-3791 (voice)
> Associate Professor
> Dept. of Statistics                              814-863-7114 (fax)
> Penn State University                         814-865-1348 (Statistics)
> University Park, PA 16802-2111