We first explored the performance of the MRF GO:BP predictions at various thresholds of t_p. In order to do this, we calculated the precision and recall of the predicted gene-GO:BP annotations with respect to the held-out gene-GO:BP annotations. It has been observed that measurements of performance on predicted GO terms tend to be quite conservative 24. This stems from the fact that gene annotation is far from being complete, and the extent to which genes are under-annotated, with as yet undiscovered pleiotropic functions, is not known. This under-annotation will lead to an underestimate of true positives and likely an overestimate of false positives, which will result in a lower measure of precision. Nevertheless, while these performance measures need to be interpreted in light of the fact that they are inherently conservative, they do provide a useful relative measure of performance. Here, a predicted gene-GO:BP annotation was called a true positive if the predicted term matched the held-out term, or the parent or child of the held-out term as defined in the GO. A predicted gene-GO:BP annotation was called a false positive if the predicted term did not match a held-out term on that gene, or a parent or child term. Lastly, a false negative was called for all held-out gene-GO:BP associations where we did not predict a term. In addition to measuring precision and recall in relation to all the held out gene-GO:BP annotations, we also measured the precision and recall with respect to the genes with held-out annotations. In this case we called a gene prediction a true positive if at least one predicted annotation for the gene is a true positive gene-GO:BP prediction. A false positive gene prediction was called if predictions were made for the gene but none were correct. Lastly, a false negative gene prediction was called if the gene had held-out GO:BP terms but we did not make a prediction for the gene.

Figure 7 shows precision (Figure 7a) and recall (Figure 7b) as a function of t_p. These plots show the general trend that increasing t_p increases the precision and decreases the recall of gene-GO:BP predictions. In contrast, precision related to gene predictions stays relatively flat over increasing t_p. This indicates that, for the predictions made for a gene, at least one has a high likelihood of being true regardless of t_p, but the likelihood that any individual GO:BP prediction is true increases with increasing t_p. We report a precision for gene-GO:BP predictions of 23% at 20% recall. This is comparable to the average of 21% precision at 20% measured over seven different groups predicting GO:BP annotations for mouse 24. While it should be noted that there are slight differences in both the input data and the way precision and recall were measured, this comparison serves to illustrate that precision of our predictions is similar to that achieved for another metazoan.

After establishing the precision and recall for predictions with the integrated networks, we address the first question of establishing a threshold on t_p that produces reliable predictions. In order to quantify the similarity between the held-out and predicted annotations in the tenfold cross-validation, we used a measure of semantic similarity (SS) and calibrated this measure against a benchmark dataset. In the context of this study, SS provides a quantification of the degree of similarity between two sets of GO:BP terms taking into consideration the structure of GO. The measure of SS was calculated using the program G-SESAME (Gene Semantic Similarity Analysis and Measurement Tool) developed by Wang et al. 73. The scale ranges from [0,1], where 0 indicates that two sets of GO:BP terms are unrelated, and 1 indicates two sets are the same. As an example, Figure 8a illustrates the overlap of two sets of terms within the structure of the GO where SS = 0.45. In order to calibrate this scale with respect to a known benchmark, we examined the distribution of SS scores between all pairs of genes with reported GIs (Figure 8b). Since GIs are reliable indicators that two genes function in a common biological process - both experimentally and also shown through the LLS - this provided a useful reference set. The median SS of gene pairs with reported GIs is 0.45, which we adopted as a reasonable cut-off for our analysis. We then used G-SESAME to measure the SS between known GO:BP annotations compared to the predicted GO:BP terms. This was performed for the tenfold cross-validation of both network sizes, 20 K and 200 K, where M = 1.8 over t_p ∈ [0, 1].

These results can be seen in Figure 8c, where the general trend shows that increasing t_p also increases the proportion of genes with predictions that have a SS > 0.45 when compared to the held-out annotations for the same set of genes.

Summaries of GO:BP predictions that were made using both and are shown in Figure 8d. We can see that at a t_p > 0.5, there are an average of 10.5 GO:BP predictions made on 941 genes for and an average of 12.7 GO:BP predictions made on 3,816 genes for . Extrapolating from the SS results shown in Figure 8c, at a t_p > 0.5 for , roughly 61% of genes have a set of GO:BP predictions with SS > 0.45, so we would expect about 574 genes (941 × 0.61 = 574) to have a set of GO:BP predictions with SS > 0.45. See Additional data files 2 and 3 for predictions from both integrated networks.

To address the second question of whether an integrated network built from all three types of data (GI, PPI, and MA) outperformed networks built from individual types of data, we evaluated the predictions in terms of precision and recall with respect to the held-out GO:BP annotations (see Materials and methods section). This was done for three networks built from the following data: fully integrated (GI, PPI and MA); GI and PPI only; and MA only. The integration of the GI and PPI only and MA only data was constructed for networks of 20 K and 200 K gene pairs using the WS framework where M = 1.8. When using the fully integrated network, increasing the value of t_p resulted in concomitantly increasing precision and decreasing recall. Comparing the results from the three different networks reveals that integrating across all three types of data, on average, outperforms the other two integrated networks (Figure 4; Figure S2 at 55). The network constructed from GI and PPI data performs better than the network constructed from MA data only for precision and recall with respect to GO:BP terms and precision with respect to genes; however, the MA integration performs better at recalling genes. These results are shown in Figure 9, where the example (t_p ≥ 0.5 at a network size of 20 K edges) is a fair representative of the entire set of evaluations. It should also be noted that we tested the precision and recall of random predictions, where a GO:BP prediction from the MRF method was replaced with a random GO:BP term at the same level in the GO hierarchy. These random predictions performed very poorly and consistently returned less than 1% for both precision and recall. These results demonstrate that the fully integrated network does, in fact, provide more reliable predictions than either of the other networks.

In order to provide a qualitative assessment of the GO:BP predictions, we manually inspected the set of predictions made on genes without experimental evidence for any GO:BP annotation. Predictions from (t_p ≥ 0.5) resulted in roughly 3,000 gene-GO:BP predictions over 941 unique genes. Of the 941 genes, we excluded 458 that could either not be localized to the v5.3 D. melanogaster genome or had at least one known GO:BP annotation (not IEA, NR, or ND). Thus, the set of gene predictions consisted of 1,148 gene-GO:BP predictions over 483 unique genes that could be localized to the genome and did not have any experimental annotation (10% of unannotated Drosophila genes).

These predictions were then examined in light of electronically inferred GO:BP terms, molecular function GO and cellular component GO annotation and also an updated version of gene annotation from v5.7 of the D. melanogaster genome. We also considered the best non-Drosophila sequence matches to the NCBI nr database, along with the respective annotations of these sequences. Over the entire set of 1,148 gene-GO:BP predictions, we found roughly 18% have supporting evidence concordant with our predictions (Additional data file 4). The next two paragraphs provide a few examples of the types of supporting relationships within this 18%.

In our set of predictions, there are several examples of well-studied genes that provide inadvertent cases of well-supported validation. For instance, there are examples of genes whose annotation was not recorded in v5.3 of the D. melanogaster genome, such as Cenp-C [FlyBase:FBgn0086697] and crossveinless [FlyBase:FBgn0000394]. Cenp-C is known to be a component of the centromere at mitotic anaphase 74, which we predicted to be involved in 'mitotic sister chromatid segregation' [GO:0000070]. Another example is crossveinless, which is known to function in bone morphogenetic protein (BMP) signaling required for wing crossvein development 7576. We correctly predicted the GO:BP terms 'imaginal disc-derived wing vein morphogenesis' [GO:0008586], 'regulation of BMP signaling pathway' [GO:0030510], 'torso signaling pathway' [GO:0008293], and 'regulation of transforming growth factor β receptor signaling pathway' [GO:0017015]; however, we also, and potentially erroneously, predicted 'blastoderm segmentation' [GO:0007350] and 'terminal region determination' [GO:0007362].

Further confirmation of prediction quality comes from unannotated genes with additional supporting evidence that is consistent with our predictions. For instance, CG5525 [FlyBase:FBgn0032444] was predicted to be involved in 'protein folding' [GO:0006457] where t_p = 1. Within the data used from v5.3 of the D. melanogaster genome, there was no experimental evidence for any GO:BP terms, but 'protein folding' [GO:0006457] was inferred from electronic annotation and this gene was also annotated with the cellular component GO term 'chaperonin-containing T-complex' [GO:0005832], inferred from sequence similarity. Additionally, the top BLAST hits (default settings) are chaperonin genes from Culex pipiens and Aedes aegypti. CG5525 is an example where the network prediction is consistent with gene function predicted from sequence similarity. As a final example, Nuf2 [FlyBase:FBgn0031886] was predicted to be involved in 'M phase' [GO:0000279] where t_p = 0.986. From the v5.3 annotations, this gene was inferred through electronic annotation to be involved in 'immune response' [GO:0006955]. However, when checked against the updated annotation of v5.7, Nuf2 was annotated with 'chromosome segregation' [GO:0007059], 'mitotic metaphase plate congression' [GO:0007080], and 'mitotic spindle organization and biogenesis' [GO:0007052], all of which are implied from a mutant phenotype. Nuf2 is an example where the prediction was validated through experimental evidence that became available after our predictions were made.

Overall, GO:BP predictions have been evaluated using precision/recall and SS in tenfold cross-validation. We then used these data to extrapolate the expected number of reasonable predictions that were made using the fully integrated networks. We have also evaluated the predictions qualitatively and shown that roughly 18% have independent evidence that supports the predictions. As a complete analysis, this suggests that the GO:BP predictions are valid.

The GO:BP predictions are based on the functional relationships drawn from the integrated gene networks. The construction of these relationships does not directly take into account any sequence-based information. Traditionally, function prediction methods have relied heavily on sequence and structural similarity 34. As a comparison, we used sequence similarity to infer GO:BP terms for the set of 483 genes for which we have made high-confidence network-based predictions. The translated proteins from these genes were used to search the NCBI nr database using BLASTp (E-value < 10^-6). All BLAST hits to Drosophila proteins were removed, matches under 40% identity were removed, then the top 10 hits were taken for each gene. Any associated GO:BP annotations (including IEA, NR, or ND) for the top ten hits were then transferred to the D. melanogaster gene. We were able to transfer GO:BP annotations for 224 of the 483 genes. Interestingly, when the GO evidence codes of IEA, ND, and NR were removed, the number of genes with any transferable annotation dropped to 98 of the 483. The D. melanogaster genes for which we predicted GO:BP terms using the integrated data appear to be in a class of genes where prediction of biological processes based solely on sequence similarity performs poorly. This is not surprising given the wide scoping meaning of biological process versus sequence features, which often reflect a molecular function, that is, kinase domain or DNA binding domain. Thus, gene prediction utilizing integrated gene networks is a complementary method to make predictions for the class of unannotated genes where traditional function prediction methods perform poorly.

GIs were downloaded as a pre-computed file from FlyBase, version FB2007_02 46. Interactions containing a gene not belonging to D. melanogaster were removed (that is, transgenic construct from D. simulans). All reported interactions (6,941) were given the same weight, a value of 1.

All PPIs - D. melanogaster-specific where possible - were downloaded from the following databases: BIND 49, DIP (version Dmela20071007) 50, DroID (September 2007) 51, BioGRID (version 2.0.32) 52, IntAct (September 2007) 53. The varying protein IDs across all datasets were mapped to v5.3 FlyBase gene identifiers. Any IDs that did not unambiguously map to a single FlyBase gene ID were removed. The union of reported interactions across all the datasets was taken. The experimental method used to detect an interaction was also considered. If a reported interaction was detected through multiple experimental methods, the most reliable method was ascribed to the interaction. The order for reliability is as follows: direct assays (that is, co-immunoprecipitation, biochemical assay) > high-confidence Y2H (high-confidence as reported in Giot et al. 54) > Y2H. In total, there were 25,408 reported PPIs among pairs of D. melanogaster proteins. These include 1,234 determined by direct assays and 24,408 Y2H interactions. The Y2H assays were subdivided into 4,590 high-confidence interactions and 19,584 positive interactions.

The following raw MA datasets were downloaded from Gene Expression Omnibus (GEO): [GEO:GSE94] 61, [GEO:GSE541] 91, [GEO:GSE442] 92, [GEO:GSE3854] 93, [GEO:GSE5430] 94, [GEO:GSE3057] 95, [GEO:GSE3069] 95, [GEO:GSE5147] 96, [GEO:GSE695] 97, [GEO:GSE3257] 98, [GEO:GSE5404] 99, [GEO:GSE6515] 59, [GEO:GSE6186] 100; [ArrayExpress:E-TABM-57] 101, [ArrayExpress:E-MAXD-6] 58; and supplemental pages De Gregorio et al. 57, Chintapalli et al. 102, and Tomancak et al. 103. These data used two distinct platforms; two channel cDNA or oligonucleotide spotted arrays, and single channel Affymetrix short oligonucleotide arrays. All data normalizations were performed in the R statistical programming environment 104. The datasets selected were required to have at least five conditions to make reliable correlation measures. We also did not use any datasets that were Drosophila cell lines.

Two channel experiments were normalized using local regression within the OLIN package 105. OLIN was run with default parameters, scaling turned on, and flagged spots were ignored for any calculations. The results of the full OLIN normalization are log-transformed ratio values for each gene on each individual MA slide.

The Affymetrix arrays were normalized using the Affy 106 and GCRMA 107 R packages. Affinities for all oligonucleotide sequences were calculated and the 'fullmodel' GCRMA normalization was run, resulting in log-transformed expression values for each probe set on each array.

All spots or probe sets were mapped to the v5.3 D. melanogaster genome assembly and annotation. Genome sequence files were downloaded from FlyBase under the FB2007_02 release 46. Primer-based platforms required two rounds of BLAST; one round to match the primers to the genome (BLASTn; E-value < 10^-2) and the second round to match the amplicon product to the genome (BLASTn; E-value < 10^-6). Physical coordinates from the forward and reverse primers were checked for strandedness and to make sure the PCR product would be under 1,000 nucleotides. The segment of DNA between the forward and reverse primers (including the primers) was taken as the amplicon product for that primer pair and searched back against the genome to ensure the amplicon did not align to any other region outside the intended segment, potentially leading to cross-hybridization. cDNA-based arrays required the cDNA sequence be aligned against the genome to test for potential cross-hybridization. Any amplicons or cDNAs with a second best BLAST hit with 80% sequence identity were flagged and removed. Unique BLAST hits mapping to exons of v5.3 annotated genes were assigned the corresponding FlyBase gene ID, otherwise the spot was flagged and removed.

Sequence files for both Affymetrix Drosophila array platforms (versions 1 and 2) were downloaded from the Affymetrix website 108. They contain a unique sequence for each probe set, which is searched (BLASTn; E-value < 10^-6) against the genome to test for potential cross-hybridization. A segment of DNA associated with a probe set was assigned a v5.3 FlyBase gene ID if the BLAST result showed a putative hit to at least one or part of one exon from one gene. A probe set was not assigned a gene ID and flagged if the BLAST result was ambiguous, meaning the second best BLAST hit was greater than 80% sequence identity, or the query sequence did not hit at least one exon.

For either MA platform, gene expression profiles were constructed using the calculated expression values for a gene across the tested conditions. If a gene expression profile had greater than 25% absent/removed expression values, that gene's profile was removed, otherwise missing values were inferred using KNNimpute 109.

We defined an MA dataset to be the full, published unit of data, and, where possible, datasets were additionally defined as the subcomponents of the published dataset. For example, the Arbeitman et al. 61 study contains six datasets; all published conditions, embryo, larva, pupa, adult male, and adult female. See Table 1 for the breakdown of all datasets.

Gene expression profiles that did not change over the course of a dataset - referred to as 'flat' - were filtered out. This was done on a gene by gene basis by taking the difference between the maximum and minimum expression values across all conditions in one dataset. For the Affymetrix platform, if the difference between the maximum and minimum expression values was less than 50, then that gene and corresponding expression profile was removed. For the two channel experiments, if the difference between the maximum and minimum log ratio value was less than.5, then the gene and corresponding expression profile was removed.

The count of genes annotated for the organisms discussed in the introduction were downloaded from the GO website 56. The annotation counts were limited to the biological process component of the GO. Additionally, the evidence codes IEA, ND, and NR were ignored.

Specific to Drosophila, gene annotations for GO:BP terms were taken from the FB2007_02 version of FlyBase 46. These data provide a mapping from a FlyBase gene ID to the GO:BP term ID(s). GO:BP terms with the following evidence codes were removed: IEA, ND, and NR. The structure of the GO is a directed acyclic graph, meaning each term has a parent term(s) (the root term is the only exception) and each term potentially has a child term(s). As described in Lord et al. 110, a connection was drawn in the ontology for the link types 'is-a' and 'part-of', then each gene was propagated from its annotated position on the GO to the root. Thus, the number of genes associated with any particular term, t_i, in the GO includes the genes annotated to t_i and additionally subsumes any genes that are annotated to the child term(s) of t_i.

It should be noted that we also evaluated two additional, potential data sources, which include matches to transcription factor binding sites 8586 and fluorescent in situ hybridizations 84; however, these data were not included as they did not meet our evaluation criteria (data not shown).

In total, 34 MA gene expression datasets were collected, normalized, and filtered. We define these 34 datasets as D = {D₁, D₂,...,D₃₄}. The Pearson correlation coefficient was calculated for all gene pairs in a dataset D_i ∈ D, For n genes in D_i = {g₁, g₂,...,g_n}, each g_j ∈ D_i is a vector of expression values across m conditions. The Pearson correlation coefficient between g_x, g_y ∈ D_i, where 1 ≤ x ≤ n and 1 ≤ y ≤ n was calculated as:

Calculating the correlation between all g_x, g_y ∈ D_i results in a distribution of correlation values. Since the majority of correlations do not reflect a functional linear relationships between two genes, only statistically significant correlations were used. Significance of the correlations were assessed through permutation testing. Within each condition of a particular dataset, gene expression values were shuffled, thus randomizing the correlation measures for each gene. From the shuffled data, 20% of the genes were selected at random and the pairwise Pearson correlation coefficient calculated for this subset of genes. This process was then repeated five times to create a stable empirical null distribution of correlation coefficients. Any correlation coefficients with a P-value < 0.01 on the two-tail null distribution - corresponding to positive and negative correlation values - were considered for further analysis.

The general procedure for integrating gene-gene relationships across all datasets was adapted from Lee et al. 1228. Datasets and the functional relationships drawn between two genes were scored in relation to GO:BP annotation, where the annotations met the same criteria as mentioned in the previous section. The LLS was calculated for each dataset as follows (we will use the same notation as Lee et al. 1228; in particular ~ denotes 'not'):

D represents a dataset of gene pairs and can be PPI, GI, or MA. I represents the set of gene pairs that were annotated and shared at least one GO:BP term, while gene pairs in ~ I were annotated, but there was no overlap between the GO:BP terms annotated to individual genes in a pair. Both I and ~ I are counts taken across all genes in the v5.3 D. melanogaster genome. P(I) is the probability of a gene pair sharing at least one GO:BP annotation, and P(~ I) is the complement. The probability of finding an annotated gene pair sharing at least one GO:BP term restricted to the gene pairs within dataset D is P(I|D), and P(~ I|D) is the complement. In the case of MA data, D represents the dataset after being filtered for significant correlation values and removing 'flat' expression profiles.

A LLS was calculated for the entire GI dataset. Each reported gene pair was weighted equally; therefore, a gene pair within the GI dataset was assigned a LLS score calculated from the entire dataset, where LLS = 2.661.

The PPI data were separated into three subsets reflecting the expected reliability of the experimental methods to detect interacting proteins. A LLS was then calculated for each subset. Protein pairs within a subset were assigned their respective LLSs. The first class of PPIs reflected interactions reported in a Y2H assay, where LLS = 0.630. The second class reflected interactions defined as high-confidence Y2H, where LLS = 1.045. The most confident class of experimental techniques (noted 'direct assay') included co-immunoprecipitation, affinity methods, biochemical assays, and mass spectrometry, where LLS = 2.389.

As described in Lee et al. 1228, gene pairs from each individual MA dataset (filtered on significant correlations and 'flat' profiles) were first ordered according to their correlation coefficients and then separated into bins of 1,000 gene pairs, where the first bin contains the most significant positively correlated gene pairs. A LLS was then calculated for each bin and plotted against the mean correlation value for bin i (Figure 2). From this plot, we fit the polynomial equation , using the lm() function in R. A separate curve was fit for both positively and negatively correlated data. Every point along the curve for a positive correlation was greater than a LLS of 0, while every curve fit to the negative correlations had at least some portion that fell below a LLS of 0. Therefore, only the significant positively correlated data were considered in evaluating each MA dataset. From all fit curves, a measure of the fraction of variance explained by the model was calculated as:

where f_i is the i^th fitted value of the model, y_i is the fitted value plus the residuals for the i^th bin, and is the average of y_i over all i bins. Additionally, the value for r² was adjusted for the number of coefficients in the model. Datasets that had an adjusted r² < 0.5 were removed from further analysis. Also, datasets were required to have a positive linear trend. After applying these criteria to all MA datasets, 20 of the 34 passed and were used in this study, whereas 14 of the 34 did not meet these criteria and were removed (Table 1; Figure S1 at 55 for all datasets). In two cases (Sorensen et al. 96 and Edwards et al. 99), all datasets related to one experiment passed the above criteria. To remove the redundancy with these two cases, the datasets constituting the subcomponents of the experiment were chosen over the full set of conditions. Specifically, the Sorensen et al. 96 control timecourse and heat-shocked timecourse were used and the dataset consisting of all conditions was not used. Within the Edwards et al. 99 datasets, two lines of flies were tested, so line 1 and line 2 were used and the full set of conditions was not used.

The positively correlated gene pairs in the 20 datasets passing the above criteria were rescored and assigned a LLS according to the fit polynomial equation. This rescoring transformed a gene pair's correlation coefficient into a LLS.

The weighted sum (WS) was adapted from Lee et al. 1228 and was calculated as follows:

LLS values for a gene pair across all k datasets were ordered from largest to smallest LLS_i ≥ LLS_i+1, ∀i; 0 ≤ i ≤ k 1, M is a free parameter and can be adjusted to increase or decrease the contribution of subsequently ranked LLSs. It should be noted that ignoring the denominator (i·M) and simply summing all LLSs across the k datasets is akin to a naïve Bayesian integration. This assumes uniform priors on each of the k datasets. Although, this method of integration is not completely Bayesian as the values being summed are LLSs and not probabilities. The opposite of ignoring the denominator is to set M → ∞. This causes the WS calculation to consider only the 0^th ranked LLS (that is, WS = LLS₀). To test a range of integration scores, WS calculations were made for all gene pairs where M ∈ {1,2,5,10,100}, M → ∞, and also for the naïve method. These seven WS calculations were selected to cover a range of different weighting schemes.

The KEGG pathways were used to validate functional relationships in the integrated network 113. To test the overlap between KEGG and GO, we compared gene-gene associations derived from KEGG pathways and the set of GO:BP annotated gene pairs used in our analysis. This comparison revealed that roughly a quarter of the gene pairs from KEGG pathways are also present as gene pairs in GO:BP.

Gene IDs for each KEGG pathway were mapped to the v5.3 genome annotation. The genes in each pathway were tested against a network through the measure of coherence. The network is a graph and can be defined as G⟨V, E⟩ with V vertices (genes) and E edges (functional relationships). The set of KEGG pathways is defined as K = {K₁, K₂,...,K_n}, where K_i is the set of genes defined by KEGG pathway K_i. The greatest connected component for K_i, noted , was determined by the greatest number of genes in K_i present and creating a connected component in G⟨V, E⟩. The coherence for K_i was then calculated as . Twenty-five pathways were selected to evaluate the WS integrated networks (Figure 3; the 25 pathways are marked with asterisks in Table S5 at 55). The 25 KEGG pathways were selected because they consistently showed the highest coherence amongst all the KEGG pathways tested.

The scores for each of the seven WS calculations were rank ordered, then networks were built starting from the top 1,000 scoring gene pairs in increasing intervals to networks of one million edges. The average coherence of the 25 pathways over each of the size intervals was measured (Figure 3). The curves in Figure 3 were then used to determine the smallest network size that provides a high overall coherence across KEGG pathways, since the average coherence varies as a function of the size of the network. We identify the points on the curve where the gain in average coherence flattens as the size of the network increases. These points of the curves occur at network sizes of 20 K and 200 K. These two network sizes are used throughout the rest of this study.

After establishing the network sizes, we aimed to optimize the M parameter in the WS score to provide the greatest average KEGG pathway coherence. Since most of the coherence was gained by the network size of 200 K gene pairs, this network was used to evaluate seven WS integration schemes. This was done by measuring the AUC. Large gains of KEGG pathway coherence in the smaller sized networks results in a higher AUC, while slow or little gain in coherence results in a low AUC. Thus, the AUC (Figure 3) is a means of assessing how well a WS integration method recovers KEGG pathway relationships. By iteratively testing networks built with increasing M values from 1, we determined the WS integration where M = 1.8 maximized the AUC for the network size of 200 K edges.

All KEGG pathways having at least ten D. melanogaster genes were tested individually against the WS network, where M = 1.8 at a size of 200 K edges. In total, 63 pathways were tested. Statistically significant coherence measures were evaluated through permutation testing; an empirical null distribution of coherence values was calculated by randomly sampling 1,000 times a set of genes equivalent to |K_i|. A single-sample Wilcoxon ranked-sum statistic was used to measure the significance of K_i when compared to the null distribution. P-values were adjusted using a Bonferroni correction.

The GO:BP predictions were evaluated using tenfold cross-validation. All genes annotated with GO:BP terms were randomly divided into ten equal sets, G = {G₁, G₂,...,G₁₀}. The following methods are performed for each of the ten sets in G. The annotations for all the genes in set G_n (where n = {1, ..., 10}) were masked from their corresponding genes. The LLS and WS integration, where M = 1.8, were recalculated for each dataset. Note that just the annotations are removed from the set of genes, but the genes remain in the analysis. The newly calculated WS relationships were rank ordered and networks with the top 20 K values and 200 K values were built. These two networks along with the GO:BP annotations from sets {G₁, ..., G₁₀}-{G_n} were then used as input to the MRF prediction method. Predictions were made on all genes in the network and measures can be used to evaluate the performance of predictions in relation to the held-out annotations for G_n.

Two methods were used to evaluate the GO:BP predictions made on the genes in G_n. First, the precision () and recall () were calculated with respect to GO:BP terms and also with respect to the genes (tp = true positives, fp = false positive, and fn = false negative). The second method measured the semantic similarity (SS) between the known set of annotations for a gene and the predicted terms for that gene.

Precision and recall with respect to the GO:BP terms were calculated as follows. A true positive prediction was called if the predicted term exactly matched a known, held-out term, or the known term's parent(s), or the known term's child(ren) (± 1 level in the GO with respect to one GO term). A false positive was called if the predicted term did not match a known, held-out term or a parent or child of the known term. A false negative was called for any known, held-out annotation not called a true positive. It should be noted that we also tested a more stringent criterion of requiring predictions to exactly match known GO:BP terms and a less stringent criterion where predictions can match ± 2 levels in the GO hierarchy. The evaluation method we used is a fair balance between the more and less stringent criteria and the precision/recall values followed the same trends for each of the three tested criteria.

A measure of precision and recall was also calculated in relation to the gene. Extrapolated from the evaluation methods of GO:BP terms, we counted a true positive gene prediction if a gene had at least one true positive GO:BP term prediction. In other words, a true positive gene was called if the intersection between previously known, held-out terms and predicted terms was at least 1. A false positive gene was called if GO:BP terms were predicted on a gene, but none matched the known, held-out terms (intersection of 0) and false negatives were called on genes that had known, held-out GO:BP terms, but a GO:BP prediction was not made on the gene.

In addition to precision and recall, we calculated SS between the set of held-out terms and predicted terms for the same gene. We employed the SS calculation developed by Wang et al. 73. Briefly, each GO term is assigned a semantic value based on the term's location in the GO hierarchy and the relationship types between ancestor GO terms 'is-a' and 'part-of'. The SS between two GO terms was calculated by considering the location of both terms in the ontology and the relationships between the ancestor GO terms jointly. SS between two sets of GO terms, which is representative of the annotations of two genes, was calculated by iteratively comparing each GO term from the held-out set to the GO terms from the set of predicted terms, and vice versa. This method calculates a single SS measure on the interval [0,1] for each annotated gene pair compared.

To determine a reliable SS threshold, we measured the SS between all reported GI gene pairs where each gene in the pair was annotated with at least one GO:BP term. GIs provided the highest LLS for any dataset and, therefore, was used as the benchmark set for SS scores. The median measure of SS for GIs was calculated to be 0.45, which we determined to be the threshold to consider a SS score reliable.

The translated protein sequences for each of the 483 genes tested were downloaded from FlyBase FB2007_02 46. The sequences were searched against the NCBI nr database using BLASTp with an E-value cutoff of 10^-6. Sequence hits with less than 40% identity were removed. Also, all sequences from the Drosophila genus were removed. The top 10 BLAST hits for each of the 483 genes were taken and the GO:BP annotations for these BLAST hits were downloaded from the GO database 56. The mapping between BLAST results and GO term annotations was done through UniProt IDs. All GO:BP annotations were directly transferred to the D. melanogaster gene from the top ten BLAST hits.

The goal of the subnetwork construction algorithm is to build a tightly connected subnetwork around a set of query genes. This was done by first defining a set of query genes, Q. This set is user defined and in this case is a set of genes that share a common biological process. We are given a graph G = ⟨V, E⟩, where v_i∈ V and v_i, v_j∈ E. In this analysis, G = and Q ⊂ V. We want to find a new set of genes, Q', that contains all v_i that meet the following criteria: v_i ∈ V, v_i ∉ Q, v_i, v_j ∈ E, v_i, v_k ∈ E, v_j ∈ Q, and v_k ∈ Q. In other words, we want to find all nodes in G that are not already present in Q and have an edge between at least two nodes in Q. This new set of nodes, Q', is then added to Q (Q = Q ∪ Q'). A second iteration of this procedure is performed to find a new set Q' in relation to Q. The two sets are again combined to form the final set Q. The subnetwork G' is returned, where G' ⊂ G and G' = ⟨Q, E'⟩, E' ⊂ E.

All genes from the wild-type muscle tissue gene expression experiment (fed versus starved larvae) were rank ordered according to their log-transformed ratio values. Gene sets were defined for the following categories: category 1, the functional categories reported in Teleman et al.; category 2, the genes from the subnetworks constructed from query seed sets from category 1; category 3, genes listed in KEGG pathways; and category 4, the three GO categories of biological process, molecular function, and cellular component. Gene sets from category 1 were taken directly from the list of genes reported in the figures of Teleman et al. Gene sets from category 2 were defined as the genes present in a seed set (gene set in category 1) in addition to the genes from the network constructed according to the subnetwork construction algorithm. Genes that were present in sets from category 1 but not found in the integrated network were not included in any sets in category 2. Gene sets from category 3 were defined by the genes in individual KEGG pathways. Gene sets from category 4 were defined by the genes annotated to individual GO terms. Gene-GO term sets were parsed directly from all associations defined by FlyBase (including IEA, NR, and ND) 56.

The GSEA 81 software was run using the 'GseaPreranked' option, with the rank ordered list of wild-type muscle expression ratios and all gene sets as input. Gene sets smaller than 15 and bigger than 500 were ignored and default weighting parameters were used.