A prediction-based resampling method for estimating the number of clusters in a dataset

Sandrine Dudoit¹^,3 and Jane Fridlyand²^,3

¹Division of Biostatistics, School of Public Health, University of California Berkeley, 140 Earl Warren Hall, Berkeley, CA 94720-7360, USA

²Jain Lab, Comprehensive Cancer Center, University of California San Francisco, 2340 Sutter St, San Francisco, CA 94143-0128, USA

³Both authors contributed equally to this work

author email corresponding author email

Genome Biology 2002, 3:research0036.1-0036.21doi:10.1186/gb-2002-3-7-research0036


Published:	25 June 2002

Subject areas: Genome studies, Bioinformatics, Methods

Abstract

Background

Microarray technology is increasingly being applied in biological and medical research to address a wide range of problems, such as the classification of tumors. An important statistical problem associated with tumor classification is the identification of new tumor classes using gene-expression profiles. Two essential aspects of this clustering problem are: to estimate the number of clusters, if any, in a dataset; and to allocate tumor samples to these clusters, and assess the confidence of cluster assignments for individual samples. Here we address the first of these problems.

Results

We have developed a new prediction-based resampling method, Clest, to estimate the number of clusters in a dataset. The performance of the new and existing methods were compared using simulated data and gene-expression data from four recently published cancer microarray studies. Clest was generally found to be more accurate and robust than the six existing methods considered in the study.

Conclusions

Focusing on prediction accuracy in conjunction with resampling produces accurate and robust estimates of the number of clusters.