Email updates

Keep up to date with the latest news and content from Genome Biology and BioMed Central.

This article has not been peer reviewed.

Open Badges Deposited research article

A tool for comparing different statistical methods on identifying differentially expressed genes

Paul Fogel1, Li Liu2*, Bruno Dumas3 and Nanxiang Ge2

Author Affiliations

1 Paul Fogel Consultant, 4 rue Le Goff, 75005 Paris, France

2 Biometrics and Data Management, Sanofi-Aventis, Mail Stop B-203A, PO Box 6800, 1041 Route 202-206, Bridgewater, NJ 08873, USA

3 Yeast Genomics, Functional Genomics, Sanofi-Aventis,13 Quai Jules Guesde, 94403 Vitry sur Seine Cedex, France

For all author emails, please log on.

Genome Biology 2004, 6:P2  doi:10.1186/gb-2004-6-1-p2

This was the first version of this article to be made available publicly.

The electronic version of this article is the complete one and can be found online at:

Received:7 December 2004
Published:8 December 2004

© 2004 BioMed Central Ltd



Many different statistical methods have been developed to deal with two group comparison microarray experiments. Most often, a substantial number of genes may be selected or not, depending on which method was actually used. Practical guidance on the application of these methods is therefore required. We developed a procedure based on bootstrap and a criterion to allow viewing and quantifying differences between method-dependent selections. We applied this procedure on three datasets that cover a range of possible sample sizes to compare three well known methods, namely: t-test, LPE and SAM.


Our visualization method and associated variability conformation rate (VCR) criterion show that standard t-test is appropriate for large sample sizes to allow accurate variance estimates. LPE borrows strength from neighboring genes to estimate the variances and is therefore more appropriate for small sample sizes whenever gene variances are similar for similar gene intensity levels. SAM has both advantages of considering gene specific variance like t-test and adjusting multiple tests by permutation based false discovery rate. However, for small sample sizes and in cases of numerous expressed genes, the distribution based on permutated datasets may not approximate the null distribution well, resulting in an inaccurate false discovery rate. Moreover, genes with low variances may be filtered because of the fudge factor.


We proposed using VCR to assess different statistical methods available for analyzing microarray data and developed a bootstrap method - on which our criterion is based - to estimate the 2-d distribution of treated vs. control gene intensity levels, under the null hypothesis that there is no difference between the treatment and control group. The biological evaluation of selected genes according to one or another method confirmed that this criterion is indeed appropriate to help identifying the most suitable method.

Additional data files

The following additional data files are provided with this article: Additional data file 1, depicting a table showing the overlap among different methods for the yeast data; Additional data file 2, showing additional Figure 1; Additional data file 3, showing additional Figure 2; Additional data file 4, showing additional Figure 3; Additional data file 5, showing additional Figure 4.

Additional data file 1. A table showing the overlap among different methods for the yeast data

Format: PDF Size: 106KB Download file

This file can be viewed with: Adobe Acrobat Reader

Additional data file 2. Additional figure 1

Format: PNG Size: 18KB Download file

Additional data file 3. Additional figure 2

Format: PNG Size: 16KB Download file

Additional data file 4. Additional figure 3

Format: PNG Size: 16KB Download file

Additional data file 5. Additional figure 4

Format: PNG Size: 17KB Download file