The principal output of MEM is a ranked list of genes that are coexpressed with the query gene in the provided datasets. For each resulting gene, MEM provides a P-value that reflects the significance of its expression similarity to the query gene across the collection on analyzed datasets. A wealth of interesting information is presented in the graphical rank matrix (Figure 1). Each column of the matrix stands for a dataset, each row represents a gene, and each matrix element reflects the individual similarity rank for the given gene in the given dataset. Visual inspection of the rank matrix allows the researcher to detect patterns of correlation across datasets and spot significantly stronger coexpression profiles. The rank aggregation algorithm provides a natural cutoff between informative and non-informative ranks for each gene. Colors and cell size is used to highlight datasets where the given gene was particularly similar to the query gene and hence contributed significantly to the final P-value.

Genes with the greatest similarity rankings are frequently in strong correlation only within a relatively small fraction of datasets that are biologically relevant to gene function. If the contributing datasets can be related in the context of experimental design, one may learn additional information about the query gene and its association to the resulting genes. Columns of the rank matrix are clustered hierarchically, so that datasets with similar correlation patterns are grouped together using a tree visualization, and datasets with most impact are aligned to the left. While the default policy is to filter datasets based on the standard deviation criterion, one may take advantage of the high contribution of few datasets and manually remove experiments that have little impact on the final list of correlated genes. Single clicks on datasets or tree nodes toggle whether selected experiments or entire experiment groups are regarded in downstream analysis.

A text mining technique called word cloud gives a compact semantic overview of a selected group of datasets through the descriptions of experimental designs. The word cloud detects keywords that are enriched in the experimental descriptions of the group, and uses different font sizes to highlight terms with strong statistical significance. One may study the experiment descriptions of single datasets and dataset clusters by moving the mouse over the dataset clustering tree.

Additional features of the tool reveal finer details of underlying data and create multiple pointers for further analysis. Besides coexpression associations in the rank matrix, MEM also displays standard heat maps with expression profiles and experimental details of individual datasets. The heat maps provide an easy visual validation of detected coexpression patterns. MEM includes filters that constrain the output to certain genes and allow the researcher to seek answers to interesting problems. For instance, one may study the association of the query gene in relation to a certain pathway or biological process, by comparing the expression patterns of its members. The URLMap feature provides easy access to external resources, as it automatically links resulting genes to multiple genomic databases 27. Coexpressed genes can be directed to the g: Profiler toolset for functional enrichment analysis of Gene Ontology terms, pathways and cis-regulatory motifs 23.

The homeobox transcription factor NANOG is a key regulator of differentiation and pluripotency maintenance in mammalian embryonic stem cells 2829. NANOG forms a complex circuitry together with the factors OCT4 and SOX2 and is involved in the combinatorial regulation of a range of downstream developmental processes.

We demonstrate the power of the MEM toolset by analyzing the genes that show strong coexpression patterns with NANOG across multiple datasets (see Figure 1). We chose a collection of 487 mouse datasets of the Affymetrix 430-2 platform, as the platform includes the largest amount of ES cells related experiments. After applying the default standard deviation filter (σ = 0.29), MEM automatically removed 419 datasets where the expression level of NANOG was insufficient for coexpression analysis. As the role of NANOG role is believed to be restricted to embryonic stem cells only, datasets covering other tissues and conditions are expectedly uninformative and provide no results of statistical significance (data not shown). On the other hand, datasets considered relevant by MEM appear to be related to the role of NANOG. Keyword analysis of experimental annotations reveals enriched terms like 'embryonic', 'pluripotent', 'stem cell' and so on (see word cloud, Figure 1a).

In response to the NANOG query, MEM retrieves a list of coexpressed genes that appear to be functionally related to embryonic stem cells. Enrichment analysis with top 50 probesets reveals important functional terms from Gene Ontology (for example, stem cell development P < 10^-12 and regulation of transcription P < 10^-6). The top list includes key transcription factors OCT4 (position 1) and SOX2 (position 7) as well as other genes with known roles in stem cell regulation and maintenance of pluripotency. For instance, UTF1 is a ES cell specific transcriptional coactivator 30, while DPPA2/3/4/5A are nuclear factors with a role in regulating pluripotency 31. NODAL is a member of the TGF-beta superfamily whose signaling is required for maintaining pluripotency in human embryonic stem cells 32. Signaling of TDGF (Cripto) in a NODAL-dependent manner directs the differentiation and fate determination of ES cells 33. TGF3 is another growth factor that has been shown to involve in the patterning of the anterior-posterior axis and exhibit signaling similar to NODAL 34.

In a previous study, Sharov et al. inferred direct targets of NANOG by computational integration of gene expression and chromatin immunoprecipitation data 35. 14 of the 281 targets of the above study are also detected by MEM among top-50 most significant genes (P < 10^-13). To put this result into context, we performed a similarity search in each of the 487 datasets individually, and found that each dataset yielded a smaller number of targets than the composite MEM query (Figure 2). To show the utility of the standard deviation based filter, we highlighted the datasets that passed the filter. Only 20 out of 487 datasets had overlap larger then 4 and only two of them did not pass the standard deviation filter, confirming the accuracy of the filter in selecting relevant datasets.

Coexpression information can be used to reconstruct biological networks and regulatory pathways 363738. In such a network, genes act as network nodes, that are associated via edges if their expression patterns are in strong correlation. Coexpression networks have been shown to contain densely connected modules that include genes of related function 10.

We used MEM to build a coexpression network of the mouse genome, using a collection of 89 datasets (Additional file 1) of the Affymetrix U74Av2 platform as the search space. In the first stage, we retrieved the list of coexpressed genes for every mouse gene, and constructed the network by connecting gene pairs where both genes of the pair had significant MEM similarity scores with one another. After applying a Bonferroni multiple testing correction, we ended up with a dense network with 115664 edges between 5440 genes with statistical significance below 0.001. In the second stage, we applied the Markov Cluster (MCL) algorithm 39 via the GraphWeb tool 40 to prune the network and find gene modules. The MCL algorithm simulates a stochastic flow in the expression graph and removes edges that are visited infrequently, resulting in a collection of densely connected groups of genes. In the third stage, we assessed the functional relevance of detected modules with GraphWeb, by finding significantly enriched Gene Ontology terms (GO), Kyoto Encyclopedia of Genes and Genomes (KEGG) and Reactome biological pathways, and cis-regulatory motifs.

The size, density and functional descriptions of the six largest modules can be seen on Figure 3a. All have strong and clear functional annotations, that is, proteasome (KEGG, P < 10^-11), mitochondria (GO, P < 10^-146), cell cycle (GO, P < 10^-50), biological adhesion (GO, P < 10^-18), immune system process (GO, P < 10^-21) and protein transport (GO, P < 10^-5). Several smaller modules with interesting functional annotations are also detected, for instance one related to T-cell generation (Figure 3b, P < 10^-12) and one related to regulation of heart contraction (Figure 3c, P < 10^-7).

Stable protein complexes are made up of several physically interacting proteins. In order to keep essential complexes intact, corresponding subunits need to have consistent expression patterns across many diverse conditions and tissues. Hence, a MEM query with a selected complex subunit should retrieve the remaining complex subunits with high ranks. Queries with different subunits are expected to retrieve similar lists of well-correlated genes whose functional role is related to that of the complex in question. In order to validate MEM performance on protein complexes, we studied the expression patterns of the essential MCM (Mini Chromosome Maintenance) complex that is conserved in eukaryotes from yeast to human. MCM is involved in the regulation of DNA replication during cell cycle, a complex multistep process that involves the cooperation of a number of proteins 41. MCM is a helicase of six subunits (MCM2-MCM7) that forms the Pre-Replicative Complex (preRC) together with the Origin Recognition Complex (ORC1-ORC6) and cell division cycle proteins (CDC6, CDC45) 42. The preRC binds to the origins of recognition on the DNA and initiates replication during the G1 phase of the cell cycle. The MCM complex acts as the licensing factor of replication, ensuring that DNA is synthesized only once per cell cycle 43. Besides initializing DNA replication, MCM also has a later role during DNA synthesis in strand elongation. The presence of the complex appears to be correlated with cell proliferation and suggests roles in cancer 444546.

We composed a compendium of 145 cancer-related microarray datasets (Additional file 2) of the human Affymetrix U133A platform from ArrayExpress to analyze the expression profiles of MCM complex subunits MCM2-MCM7. For each of the MCM subunits, we used MEM to retrieve a ranked list of 100 probesets with most correlation relative to the subunit, referred to its cohort. In case of multiple probesets corresponding to a subunit, we picked the probeset whose cohort contained most cell cycle related genes. We excluded MCM7, as the corresponding probeset also maps to several unrelated genes.

The subunits of the MCM complex have extremely consistent expression profiles across the compendium of cancer-related datasets. Among the cohorts of MCM subunits, other MCM probesets are always delivered with a high rank (median rank 17.5). The MCM cohorts are generally very similar, as on average, a pair of MCM subunits shares 65 probesets of the 100-element cohorts and the six 100-probeset cohorts contain a total of 116 probesets that occur in more than two cohorts (Additional file 3). These overlaps are very unlikely to occur by random chance, as even the protein pair with least common probesets has a highly significant P-value (MCM5 and MCM6, 47 common probesets, P < 10^-87).

MEM coexpression patterns are functionally well reflected in the cohorts. The probesets have strong enrichments that are related to the role of the MCM complex as well as the cancer-specific context of the analyzed datasets. g: Profiler reveals enrichments of generic terms such as the cell cycle (GO, P < 10^-42) and DNA replication (GO, P < 10^-37), as well as more specific functions like DNA replication pre-initiation (Reactome, P < 10^-11) and DNA strand elongation (Reactome, P < 10^-21). The promoters of coexpressed genes have enrichments for the binding site of E2F1, a transcription factor with a recognized role in replication regulation and oncogenesis (for example, Transfac, M00427, consensus sequence TTTSGCGS, P < 10^-6) 4748. The enrichment in the P53 pathway (KEGG, P = 10^-4) suggests a link with the well-identified tumor suppressor gene 49. Moreover, the cohorts contain microRNAs as well as enrichments for microRNA target sites that may have cancer-specific roles. For instance, the coexpressed genes have a greater than expected proportion of target sites for the microRNA miR-142-5p (miRBase, P < 10^-4), a regulatory RNA that has been detected in the context of leukemia 50.

In order to investigate the advantage of MEM analysis for coexpression over multiple datasets, we conducted a computational experiment where varying numbers of datasets were incorporated for delivering MCM cohorts (Figure 4). For each of the sample sizes ranging from 2 to 125, we used 300 randomized collections of input datasets from the above cancer compendium to measure the median distance between MCM subunits in individual cohorts. As expected, adding more datasets into MEM analysis brings MCM subunits closer in resulting ranked gene lists. According to the Kolmogorov-Smirnov one-sided test, using MEM queries over several datasets always gives significantly better results (for example, increased similarity between MCM subunits) than correlation over any of the datasets individually. The advantage of MEM analysis appears to increase exponentially in relation to analyzed datasets. Importantly, the MEM query over all 145 cancer-specific datasets provides a smaller median distance between MCM subunits (m = 17.5), compared to the correlation over the concatenation of corresponding datasets (m = 22.5).

Rank aggregation is the heart of MEM coexpression analysis. It uses the statistical distribution of orderings to integrate individual lists of similar genes into final lists with significance P-values for each gene. The rank aggregation problem has been studied mainly in the context of voting and social choice, but there are also several bioinformatics applications, for example, 5152.

Most classical methods assume that each individual ranking is reasonable and should be taken into account in composing the final ordering. However, in the case of gene coexpression analysis, some rankings include considerable amounts of noise as they are derived from genes and conditions with low variation. In order to overcome this, we first identify reliable gene lists that are based on sufficient variation, and then compute the rank aggregation based on the limited set of lists.

The input of rank aggregation is a collection of ordered lists, where every element in a list corresponds to a gene in a specific experiment, showing the rank of similarity to the query gene g*, relative to all other genes in the organism. We normalize the lists into the range [0.1], by dividing each individual rank by the maximal rank, that is, the number of genes in the microarray platform. We transform the ranks so that for each gene g_i, we have a rank vector r(g*, g_i) = [, ..., ] where corresponds to the position of g_i in the query on dataset j.

A straightforward solution for rank aggregation involves reordering the genes g_i based on their arithmetic means of individual ranks r(g*, g_i). Unfortunately this approach is rather sensitive to noise, since the mean is heavily influenced by large ranks that indicate no strong correlation. Geometric mean is more sensitive to small ranks and robust to fluctuations among large uninformative ranks. An alternative and empirically more successful approach uses trimmed mean that only considers k smallest elements, but requires the estimation of the parameter k.

We developed a statistical strategy for robust rank aggregation that overcomes the problems of mean-based methods and allows us to evaluate the statistical significance of detected similarity. As a null hypothesis, we consider a model ranking where similar genes are permuted randomly and the distribution of each rank vector r(g*, g_i) is approximately uniform. In the biological case of strong coexpression, we observe an unexpectedly large amount of small ranks between genes with correlated expression patterns, so that the distribution of r(g*, g_i) is skewed towards small values and significantly different from a uniform distribution. We can reorder the rank vector r(g*, g_i) increasingly to gain the vector of order statistics which range from the smallest to the largest value of r(g*, g_i). Assuming the null hypothesis, we can use the binomial distribution to calculate the probability that k or more ranks are smaller than , for every k:

The final similarity score ρ between g* and g_i is defined as follows:

In other words, for every value of k, we compute the P-value for each rank statistic r_(k) being randomly as small as observed in the dataset, and as a final score we use the minimal P-value.

The final ρ score itself is not a P-value, since it is a minimum of P-values. Still, we may use a multiple testing correction to remove false positives that occur due to several independent tests. As we calculate the ρ scores for each gene, we actually find a P-value corresponding to each rank matrix element. According to Bonferroni correction for multiple testing, an individual P-value is significant if it is smaller than the desired significance level after multiplication by the number of rows and columns of the rank matrix. We cannot use any less stringent criteria for correction, since P-values for the same gene are strongly correlated.

As a byproduct of the above computation, we gain information about the datasets that contain significant coexpression between any two genes. A dataset with a ranking that is smaller than the ranking that gave rise to ρ(g*, g_i) can be considered significant. This feature allows us to highlight the contributions of different datasets into the final similarity ranking, and observe interesting patterns between related datasets. The score ρ also has the advantage of being non-parametric, as it makes no requirements on the number of input datasets or the magnitude of relevant ranks. In a way our ρ-score represents a natural balance between two scenarios: a gene that strongly correlates with the query gene in a small number of samples, and a gene that shows weak correlation in a large range of samples.

All data used in the analyses has been obtained from ArrayExpress and it also includes datasets that were originally submitted to GEO. We only included Affymetrix datasets where raw data was available, and performed a uniform Robust Multi-array Average (RMA) normalization 53 with the Bioconductor affy package 54 using the default parameters. MEM also includes biological annotations of the datasets as annotated according to the Minimum Information About a Microarray Experiment (MIAME) standard 1. The annotations are used for building word clouds and annotation tracks in heat map visualization of gene expression data.

We performed a simulation study to find the threshold for query gene variation that would best identify the datasets where the gene has meaningful expression patterns. All the experiments in MEM are normalized and preprocessed the same way, so we may compute a uniform threshold that applies to all datasets. In the simulation, we chose random sets of 2000 genes and 140 experiments on human Affymetrix platform HG-U133A, and calculated the standard deviation for each gene in each experiment. We also performed a MEM query with each of the genes and used similarity score cutoff that yielded on average 20 genes per query. Now we tried several thresholds for the standard deviation and in each case we calculated correlation between the number of experiments exceeding the threshold and the number of genes in the result of the query. We achieved strongest coexpression patterns between the query genes and the resulting genes when using a standard deviation cutoff between 0.25 and 0.39, while the peak performance was observed at the threshold 0.29 (Additional file 4).

MEM uses word clouds to display aggregated annotations of multiple datasets. As a first step in generating the word clouds, we process textual annotations of each dataset to extract words and multi-word expressions. Out of all the words present in the dataset description we pick only nouns, adjectives and some other matching predefined patterns. Selected words are then normalized to ignore inflected forms (for example, gene, genes) using WordNet lemmatiser 55. Besides single words, we also extract noun and adjective phrases. Syntactic analysis is performed using MedPost part-of-speech tagger 56.

Next, for a given group of datasets, we figure out a set of descriptive terms (words and phrases) that are over-represented in this group, compared to all the available datasets. We use hypergeometric P-value to identify such group-specific terms. The word cloud is then composed out of the terms with the lowest P-value. Within the word cloud, font size depicts their extent of over-representation of the term in the corresponding group of datasets.