[BioC] discrepancy between limma and samr due to difference in fdr adjustment?

[Johan Lindberg]

Dear all.

I have used Limma to analyse my data and I got no differentially  
expressed genes correcting with multiple testing using fdr. Then a  
colleague of mine analysed the data and used samr, the package by the  
guys from Stanford, http://www-stat.stanford.edu/~tibs/SAM/ and got  
~600 genes with an fdr < 0.05.

I immediately thought I had done something wrong in my Limma analysis  
and double checked a zillion times but I couldn't find any errors.  
When I compared the ranking for the 1000 most differentially  
expressed genes of Limma and samr they look very similar with few  
discrepancies. I attached a picture of the ranking. Then I compared  
the t-scores for the same genes and they were also almost the same. I  
also attached an image of that. The scores for Limma are a little bit  
more significant.

Its when I do the adjustment for multiple testing that I get  
differences (I attached another picture). As I understand it the fdr  
level for a certain delta-cutoff is in samr approximated by balanced  
permutations of the two groups. Thereby one can find out the median  
number of differentially expressed genes comparing the two groups in  
e.g. 100 permutations which is the fdr for that level. In Limma the  
general fdr definition by Benjamini & Hochberg is used if I  
understand it right.

I was really surprised that I got so different results using the same  
correction for multiple testing, whereas one is based on permutation  
of the same data and the other is based on the old Benjamini  
definition. What is more correct in this situation? I would really  
appreciate if someone could give me some advice.

Best regards,

Johan










sessionInfo()
R version 2.7.0 (2008-04-22)
i386-apple-darwin8.10.1

locale:
sv_SE.UTF-8/sv_SE.UTF-8/C/C/sv_SE.UTF-8/sv_SE.UTF-8

attached base packages:
[1] splines   tools     stats     graphics  grDevices utils     datasets
[8] methods   base

other attached packages:
  [1] samr_1.25            impute_1.0-5         AnnBuilder_1.18.0
  [4] XML_1.95-2           siggenes_1.14.0      multtest_1.20.0
  [7] survival_2.34-1      KTH.hsOligo.db_1.0.0 hsOligo_2.0.1
[10] kth_1.2.1            geneplotter_1.18.0   lattice_0.17-6
[13] aroma_0.94           R.io_0.37            R.graphics_0.42
[16] R.colors_0.5.3       R.basic_0.49         aroma.light_1.8.1
[19] R.utils_1.0.2        R.oo_1.4.3           R.methodsS3_1.0.1
[22] limma_2.14.5         annotate_1.18.0      xtable_1.5-2
[25] AnnotationDbi_1.2.2  RSQLite_0.6-9        DBI_0.2-4
[28] Biobase_2.0.1

loaded via a namespace (and not attached):
[1] KernSmooth_2.22-22 RColorBrewer_1.0-2 grid_2.7.0


*********************************************
Johan Lindberg
Royal Institute of Technology
AlbaNova University Center
Stockholm Center for Physics, Astronomy and Biotechnology
School of Molecular Biotechnology
Department of Gene Technology
Visiting address:
Roslagstullsbacken 21, Floor 3
106 91 Stockholm, Sweden
Delivering address:
Roslagsvägen 30 B
104 06 Stockholm, Sweden
Phone (office) +46 8 553 783 44
Fax + 46 8 553 784 81
http://www.ktharray.se/
http://www.arrayadvice.se/
*********************************************