[BioC] decideTests with nestedF

Pedro López Romero plopez at cnic.es
Mon Jun 19 13:49:12 CEST 2006


Thank you very much to all of you for the many comments and suggestions.
They have been of great help and I think that I understand much better now
the differences between the different methods.- 

I have compared the genes that I have obtained using *separate*, *nestedF*
and *hierarchical* and there is a lot of agreement. Of course, there are
some genes that are specifically selected by *nestedF* that have a high
p-value in the *separate* list, but now I think that I understand why. 

Again, thank you very much.- 


-----Mensaje original-----
De: Gordon Smyth [mailto:smyth at wehi.EDU.AU] 
Enviado el: sábado, 17 de junio de 2006 6:01
Para: Pedro L?pez Romero
CC: bioconductor at stat.math.ethz.ch
Asunto: [BioC] decideTests with nestedF

At 08:00 PM 16/06/2006, bioconductor-request at stat.math.ethz.ch wrote:
>Date: Fri, 16 Jun 2006 11:30:18 +0200
>From: Pedro L?pez Romero <plopez at cnic.es>
>Subject: [BioC] decideTests with nestedF
>To: <bioconductor at stat.math.ethz.ch>
>Content-Type: text/plain
>Dear list,
>  I have an experiment with several contrasts associated to different
>experimental conditions for each gene, so I want to select genes in a first
>step by their moderated-F statistic in order to reduce the number of genes
>for multiple contrasts across genes.
>For this purpose, I am using decideTests with nestedF, as it is said in
>limma user guide (chapter 10, p 51). However, when I use decideTests with
>method="nestedF", I get quite a few genes that have an adjusted p value in
>their separate t-tests contrast greater than 0.9. These genes have a really
>small M value as to accept that they are really differentially expressed
>between conditions. Does this make sense or I am doing something wrong?.-
>The adjusted pvalue that I am talking about is the one that I see after
>applying the topTable function for the corresponding single contrast
>two experimental conditions. In this case, the adj.P.Val is the same as the
>ones in decideTests with method ="separate", since we are applying the same
>My code is:
>topTable(fit2,coef=i,number=100 adjust="fdr")
>Would be the function p.adjust(fit2$F.p.value,method="BH") (as Gordon
>proposed in a recent email about this topic) more appropriate in order to
>what I what to do?
>Any comment will be of great help to me.
>Thanks a lot.-

Dear Pedro,

Different adjustment methods do give different results. It is as 
simple as that. The phenomenon that you have observed is perfectly 
possible and quite intentional. (Surely this is not surprising. 
Afterall, if different adjustment methods gave the same result, then 
there wouldn't be different methods.)

Please understand that nestedF does not give any formal control of 
FDR at the contrast level. The only control is at the probe level: by 
the above code, you have controlled the expected proportion of genes 
falsely identified to have *any* significant contrasts to be less than 0.05.

If you are seeking control of FDR at the contrast level, you should 
probably use another adjustment method. The methods "global" and 
"separate" can both give control of FDR at the contrast level. The 
strategy which you seem to want to do, using F-tests first and then 
t-tests for the significant genes, with control of FWER or FDR at 
each level, is called method="heirarchical" in limma.

Please understand also that control of FDR behaves differently from 
control of the family-wise error rate (FWER), which is the 
traditional emphasis of multiple testing. The idea of hierarchical 
testing with F-tests before t-tests is far more effective for 
controlling FWER than it is for controlling FDR. It doesn't make much 
difference for FDR.

The real motivation for nestedF is to discover genes which respond 
simultaneously to several contrasts. You will find that the gene 
which is worrying you so much has several significant contrasts. 
nestedF is designed to become less and less rigorous as it works down 
the contrasts for a given gene. As I said before, this is quite 
intentional. It produceds a method which is better at finding 
simultaneously contrasts but worse at finding genes which respond to 
only one contrast.

Hope this helps.
Best wishes

More information about the Bioconductor mailing list