This article is part of the supplement: EGASP '05: ENCODE Genome Annotation Assessment Project
EGASP: the human ENCODE Genome Annotation Assessment Project
- Equal contributors
1 Centre de Regulació Genòmica, Institut Municipal d'Investigació Mèdica-Universitat Pompeu Fabra, E08003 Barcelona, Catalonia, Spain
2 European Bioinformatics Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SD, UK
3 Center for Integrative Genomics, University of Lausanne, Switzerland
4 University of Geneva Medical School and University Hospitals of Geneva, 1211 Geneva, Switzerland
5 Department of Genetics, University of Cambridge, Cambridge CB3 2EH, UK
6 South African National Bioinformatics Institute (SANBI), University of Western Cape, Bellville 7535, South Africa
7 Affymetrix Inc., Santa Clara, California 95051, USA
8 Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge CB10 1SA, UK
9 Department of Molecular and Cellular Biology, University of California, Berkeley, California 94792, USA
10 Omicia Inc., Christie Ave., Emeryville, California 94608, USA
11 Member of the EGASP Organizing Committee
12 Member of the EGASP Advisory Board
Genome Biology 2006, 7(Suppl 1):S2 doi:10.1186/gb-2006-7-s1-s2Published: 7 August 2006
We present the results of EGASP, a community experiment to assess the state-of-the-art in genome annotation within the ENCODE regions, which span 1% of the human genome sequence. The experiment had two major goals: the assessment of the accuracy of computational methods to predict protein coding genes; and the overall assessment of the completeness of the current human genome annotations as represented in the ENCODE regions. For the computational prediction assessment, eighteen groups contributed gene predictions. We evaluated these submissions against each other based on a 'reference set' of annotations generated as part of the GENCODE project. These annotations were not available to the prediction groups prior to the submission deadline, so that their predictions were blind and an external advisory committee could perform a fair assessment.
The best methods had at least one gene transcript correctly predicted for close to 70% of the annotated genes. Nevertheless, the multiple transcript accuracy, taking into account alternative splicing, reached only approximately 40% to 50% accuracy. At the coding nucleotide level, the best programs reached an accuracy of 90% in both sensitivity and specificity. Programs relying on mRNA and protein sequences were the most accurate in reproducing the manually curated annotations. Experimental validation shows that only a very small percentage (3.2%) of the selected 221 computationally predicted exons outside of the existing annotation could be verified.
This is the first such experiment in human DNA, and we have followed the standards established in a similar experiment, GASP1, in Drosophila melanogaster. We believe the results presented here contribute to the value of ongoing large-scale annotation projects and should guide further experimental methods when being scaled up to the entire human genome sequence.