[BioC] decideTests with nestedF
smyth at wehi.EDU.AU
Sat Jun 17 06:01:15 CEST 2006
At 08:00 PM 16/06/2006, bioconductor-request at stat.math.ethz.ch wrote:
>Date: Fri, 16 Jun 2006 11:30:18 +0200
>From: Pedro L?pez Romero <plopez at cnic.es>
>Subject: [BioC] decideTests with nestedF
>To: <bioconductor at stat.math.ethz.ch>
> I have an experiment with several contrasts associated to different
>experimental conditions for each gene, so I want to select genes in a first
>step by their moderated-F statistic in order to reduce the number of genes
>for multiple contrasts across genes.
>For this purpose, I am using decideTests with nestedF, as it is said in
>limma user guide (chapter 10, p 51). However, when I use decideTests with
>method="nestedF", I get quite a few genes that have an adjusted p value in
>their separate t-tests contrast greater than 0.9. These genes have a really
>small M value as to accept that they are really differentially expressed
>between conditions. Does this make sense or I am doing something wrong?.-
>The adjusted pvalue that I am talking about is the one that I see after
>applying the topTable function for the corresponding single contrast between
>two experimental conditions. In this case, the adj.P.Val is the same as the
>ones in decideTests with method ="separate", since we are applying the same
>My code is:
>Would be the function p.adjust(fit2$F.p.value,method="BH") (as Gordon
>proposed in a recent email about this topic) more appropriate in order to do
>what I what to do?
>Any comment will be of great help to me.
>Thanks a lot.-
Different adjustment methods do give different results. It is as
simple as that. The phenomenon that you have observed is perfectly
possible and quite intentional. (Surely this is not surprising.
Afterall, if different adjustment methods gave the same result, then
there wouldn't be different methods.)
Please understand that nestedF does not give any formal control of
FDR at the contrast level. The only control is at the probe level: by
the above code, you have controlled the expected proportion of genes
falsely identified to have *any* significant contrasts to be less than 0.05.
If you are seeking control of FDR at the contrast level, you should
probably use another adjustment method. The methods "global" and
"separate" can both give control of FDR at the contrast level. The
strategy which you seem to want to do, using F-tests first and then
t-tests for the significant genes, with control of FWER or FDR at
each level, is called method="heirarchical" in limma.
Please understand also that control of FDR behaves differently from
control of the family-wise error rate (FWER), which is the
traditional emphasis of multiple testing. The idea of hierarchical
testing with F-tests before t-tests is far more effective for
controlling FWER than it is for controlling FDR. It doesn't make much
difference for FDR.
The real motivation for nestedF is to discover genes which respond
simultaneously to several contrasts. You will find that the gene
which is worrying you so much has several significant contrasts.
nestedF is designed to become less and less rigorous as it works down
the contrasts for a given gene. As I said before, this is quite
intentional. It produceds a method which is better at finding
simultaneously contrasts but worse at finding genes which respond to
only one contrast.
Hope this helps.
More information about the Bioconductor