We used a sparse multivariate statistical approach to relate disease phenotype to microbiome structure and function while accounting for potential correlates and confounding factors such as treatment or smoking. Metadata features potentially associated with each clade were first selected using boosting, and the significance of these associations was then assessed using a multivariate linear model with false discovery rate correction (see Materials and methods). We first investigated the resulting association of microbial clades with IBD and with features of our cohorts, testing all available metadata and clades from the genus to phylum levels. Ordination of overall relationships among samples and host status revealed several major combinations of environmental factors that co-varied with the microbiome (Figure 1; Additional file 1). For example, UC covaried in this population with mesalamine treatment, whereas CD patients were more often assessed by biopsy, treated with immunosuppressants, and enriched for Escherichia. Similarity among microbiome compositions in disease subtypes reflects those previously observed 3435, with ileal CD (iCD) representing a strong outgroup, UC a generally less-extreme microbial phenotype (less dissimilar from healthy subjects), and non-iCD a broad distribution of microbiome configurations.

An important consideration that informed the remainder of our analysis, and which is often overlooked in studies of the microbiome, was the consistent covariation among disease status, aspects of subject environment, and microbiome structure. For example, the factor most associated with changes in microbiome composition was not disease but whether the sample origin was stool or biopsy. Biopsy location induced minor changes in microbiome composition (Additional files 2, 3 and 4) relative to the extreme differences between stool and biopsy communities, in agreement with previous studies 3637. In this cohort, iCD was always represented by biopsy, whereas 18.4% of non-iCD and 36% of UC samples were biopsies. iCD was also associated with greater likelihood of immunosuppressant treatment: iCD, non-iCD, and UC patients were treated by immunosuppressants in 74.4%, 19.2% and 16% of samples, respectively. In contrast, non-iCD and UC cases were more likely to be treated with mesalamine or antibiotics: mesalamine was used for 30.2% of iCD samples, 69.2% of non-iCD samples, and 77.3% of UC samples, while antibiotics were used in 2.3% of iCD, 17.9% of non-iCD, and 13.3% of UC samples. These associations lead to a range of non-independent covariates. Although disease activity may influence microbiome composition, after adjusting for the other factors, it was not independently associated with a specific shift in the microbiome composition in our analysis, and there were no significant (P < 0.01) associations between microbiome composition and gender (Additional file 5).

The second largely independent factor influencing microbiome composition was age, itself negatively associated with smoking (Figure 1; Additional file 1). Twenty-four (10.4%) of the available subjects were less than 18 years of age and 26 were 60 years or older. Aging is associated with continual changes in the microbiome, primarily a gradual decrease in Bifidobacterium as observed here (Additional file 6) and by others 3839. After observing these overall patterns of covariation among disease, treatment, environment, and gut microbiome composition, we continued our analysis only after assessing the significance of microbiome-disease associations in a multivariate manner to account for host environment and treatment.

After adjusting for these covariates, we determined microbial clades differing significantly in abundance between healthy and IBD subjects (Figure 2a; Additional file 1). This considered age, smoking, and treatment factors (immunosuppressant, corticosteroids, mesalamine, antibiotics), as well as disease activity at sampling and sample type (stool or biopsy). Two genus-level phylotypes, Roseburia and Phascolarctobacterium, were significantly reduced in both UC and CD, while Clostridium increased, all with false discovery rate q < 0.2. Roseburia is a clade XIVa Clostridia and thus associated with anti-inflammatory regulatory T cell production in the gut 40. Cultured Roseburia have been described as acetate utilizers and butyrate producers 41, while cultured Phascolarctobacterium are exclusively succinate consumers, and produce propionate when co-cultured with Paraprevotella 42. Thus, an IBD-associated decrease in Roseburia and Phascolarctobacterium may reflect a decrease in butyrate and propionate production.

The Ruminococcaceae, which are acetate producers 43, were decreased in CD, while the Leuconostocaceae, which produce acetate and lactate 44, were decreased in UC. The only major clade with a significant increase in abundance specific to CD was the Enterobacteriaceae, specifically Escherichia/Shigella. This family has been previously implicated in intestinal inflammation 6454647.

In CD patients with ileal involvement, sequences of the Ruminococcaceae family and of Faecalibacterium in particular were dramatically reduced compared to other subjects (Figure 2a), confirming previous studies 4849. Faecalibacterium prausnitzii, the only cultured representative of Faecalibacterium, is able to metabolize both diet-derived polysaccharides and host-derived substrates such as N-acetyl glucosamine from intestinal mucus 50. It is also a major butyrate producer and exhibits anti-inflammatory effects in a colitis setting 51. The Ruminococcaceae represent the first step of microbiome-linked carbohydrate metabolism, as they degrade several types of polysaccharides in the lower GI tract, including starch, cellulose, and xylan 21. The Roseburia genus, which is significantly reduced in all IBD patients (including iCD), and the Ruminococcaceae are further functionally connected in that the latter consume hydrogen and produce acetate that can be utilized by Roseburia to produce butyrate 4143. Consistent reductions in all of these clades may thus have functional consequences on the ability of the host to repair the epithelium and to regulate inflammation.

The genera Escherichia/Shigella (indistinguishable as a 16S-based phylotype) were particularly highly enriched in iCD (q < 0.2; Additional file 1) above their general overabundance in CD patients. Lipopolysaccharide produced by Gram-negative bacteria such as Escherichia coli is a canonical microbe-associated molecular pattern, known to activate toll-like receptor 4 (TLR4) signaling 52 and thus trigger inflammatory cascades. TLR4 expression is highly up-regulated in the intestinal epithelium of IBD patients 53, and mutations in TLR4 are associated with both CD and UC 54. Previous culture-based studies have found that E. coli, specifically E. coli exhibiting pathogen-like behaviors such as adhesion and invasiveness 55, are more frequently cultured from iCD biopsies, and culture-independent studies have found an enrichment in E. coli that contain virulence-associated genes in iCD 6. This suggests that CD-involved ileum is a favorable milieu for establishment of E. coli with pathobiont features, which may have implications for IBD exacerbations and its chronicity. An inflamed ileum may furnish a specialized niche permissive for microbes with enhanced fitness in inflamed conditions.

The most severe form of UC is pancolitis, in which UC affects the entire colon; this condition is associated with greatly increased risk of colon cancer 56. Patients with pancolitis did not harbor a clear specificity in their dysbiosis. However, both these patients and iCD patients had a reduced abundance of the Odoribacter genus, which belongs to the Porphyromonadaceae family and to the Bacteroidetes phylum. As Odoribacter splanchnus is a known producer of acetate, propionate, and butyrate 57, decreased Odoribacter may affect host inflammation via reduced SCFA availability.

In the process of identifying microbiome perturbations specific to IBD, our multivariate model simultaneously analyzed the surprisingly diverse effects of environmental and treatment factors on GI microbial communities (see selection in Figure 3; complete data in Additional file 1). We observed a significant correlation between increasing age and decreasing Bifidobacterium (Additional file 6). The Firmicutes phylum also significantly decreased while Bacteroides increased with age in this cohort (Additional file 1); this agrees with previous studies 3839 and potentially reflects dietary or body mass-related changes with increasing age, which were not directly measured in these subjects, or host metabolism modifications 58.

Critical to determining causality in links between IBD and the gut microbiome, IBD treatments were also associated with alterations in microbiome composition. Mesalamine (5-aminosalicylic acid) is a bowel-specific aminosalicylate drug. Although its exact mode of action is unknown, it is thought to act as an antioxidant and to decrease intestinal inflammation, in part by peroxisome proliferator-activated receptor-γ (PPARγ) activation and inhibition of NFκB and pro-inflammatory eicosanoid production. Here, its use was linked to strong reductions in Escherichia/Shigella (> 100% of average abundance, q < 0.04; Additional file 7), in agreement with a recent study 59. Both 5-aminosalicylic acid and immunosuppressant treatment were associated with modest increases in Enterococcus, the only genus perturbed in immunosuppressant-treated patients with low false discovery rate (also > 100% of average abundance, q < 0.09).

Antibiotics were among the strongest factors associated with a reduction in ecological diversity (Figure 2b). Many individual clades were greatly reduced or nearly absent after administration of antibiotics, including the Collinsella, Dorea, Butyricicoccus, Subdoligranulum, and Acetivibrio (all q < 0.2; Additional file 1). These genera are predominantly from the Clostridiales order, Gram-positive and anaerobic bacteria that are targeted by the antibiotics commonly used in IBD, such as ciprofloxacin and metronidazole.

Smoking is likely the best-known environmental factor that impacts IBD 60. It is associated with increased risk of CD and is conversely protective towards developing UC 61. The only common organism to which tobacco usage was linked in these individuals was Anaerostipes (Firmicutes phylum), which decreased (> 60% average abundance, q < 0.15; Figure 3) in current or former tobacco users, beyond any change due solely to smokers' higher average age. The Anaerostipes genera can utilize lactate to produce butyrate 62, which is beneficial to colonic health.

Finally, as previously mentioned, samples of the stool as opposed to mucosal biopsies differed strongly in microbiome composition (Additional file 2). More than 70 clades were significantly over- or under-enriched in stool samples relative to biopsies at q < 0.2. This effect extended to entire phyla, as the Firmicutes were approximately twofold more abundant in stool (Additional file 1). Microbial habitat dictates the composition of microbial communities 36; in the GI tract, this has been suggested to occur on biogeographical scales of intestinal regions 3763 or even millimeters apart 6465, and luminal/mucosal differences may be further perturbed by bowel preparation prior to colonoscopy 66. The data did not suggest that the luminal and mucosal communities were independent; rather, all 14 clades significantly associated with IBD retained the same trend when stratified by sample origin (Additional file 8). The fecal microbiome appeared to convey a consistent but numerically transformed function of mucosal communities, both of which shifted in composition in association with host environment, treatment, and disease.

In a closer analysis of intestinal biogeography as reflected by biopsies drawn from distinct regions, differences in most clades were modest and correlated largely with previously described changes in pH (Additional files 2, 3 and 9) 67. The clades with the largest regional changes included the Roseburia and Ruminococcaceae, with lower abundance in the low-pH terminal ileum, transverse, and right colon; Alistipes, following a similar pattern; and the Fusobacteria and Enterobacteriaceae, with an opposite pattern of somewhat increased abundance in the ileum and right colon. Particularly as the former have also been associated with the colorectal cancer microenvironment in previous work 6869, it is of note that these variations in the microbiota with respect to biogeography and pH are similar to those we observed with respect to IBD and potentially redox status as detailed below.

We continued our analysis by combining community composition with over 1,200 annotated genomes from the Kyoto Encyclopedia of Genes and Genomes (KEGG) catalog 70. The genes annotated within each available reference genome were used to provide an approximate gene catalog for each community (see Materials and methods), which we reconstructed into metabolic pathways (Figure 4) and smaller modules and biological processes (Figure 5; Additional file 10) as previously described 71. Pathway, module, and process abundances were then associated with disease and host environment using the same sparse multivariate model with which microbial abundances were assessed (Additional files 11, 12 and 13).

Considering only the contrast between IBD (CD or UC) and healthy subjects, 24 of 200 (12%) total metabolic modules were differentially abundant at q < 0.2. This is in stark contrast to the microbial shifts discussed above, in which only 6 of 263 (2%) genus-level clades reached this significance threshold. Even in the absence of metagenomic or metatranscriptomic data and only leveraging the genes and pathways in reference genomes associated with these communities, changes in microbial function were more consistent than changes in community structure. This has been noted in environmental communities 72 and suggested with respect to obesity and other biometrics 7374, but to date it has not been reported for disease-linked dysbioses or IBD.

We validated these functional shifts by shotgun metagenomic sequencing of the small subset of available samples with appropriate stool DNA, seven healthy controls and four CD patients (Additional file 14). These were sequenced to a shallow depth averaging 119 meganucleotides per sample of 150-nucleotide paired-end Illumina MiSeq reads, reducing our effective limit of detection but otherwise providing close agreement with inferred metabolic shifts in the IBD metagenome. Of the modules highlighted below and in Figure 5, one (cobalamin biosynthesis) fell below the limit of detection, and the remaining six retained the expected trend of over- or under-enrichment in Crohn's disease, as did additional processes detailed below, including glycolysis and bacterial secretion.

We observed that even basic GI microbiome metabolism was altered in both UC and CD. Amino acid metabolism showed major perturbation: genes for the metabolism and biosynthesis of nearly all amino acids (particularly histidine and lysine) decreased in abundance (Figure 4), while arginine, histidine, and lysine transport (Figure 5) gene abundance increased. In iCD we also observed a decrease in glutamine-related functional modules, which would lead to a lower amount of glutamate required for gamma-aminobutyric acid, ornithine, and arginine biosynthesis; abundance of all three of these modules also decreased. In marked contrast to the other amino acids, genes for metabolism of the sulfur-containing amino acid cysteine significantly increased in abundance, with even greater increase in iCD. This corresponded with an overrepresentation of genes related to sulfate transport in UC and CD (Figure 5), and in increase in sulfur and nitrogen metabolism in CD (Figure 4).

CD was associated with increased abundance of many genes related to carbohydrate transport (Figure 5). There were large increases in pentose phosphate pathway and fructose/mannose metabolism gene abundance in iCD (Figure 4), which were accompanied by increase in carbohydrate metabolism, but they were not significant in UC and CD. In addition, iCD showed increased abundance of transporter genes for glucose, hexoses, maltose, and mono-, di-, and oligosaccharides (Figure 5). We observed a decrease in both butanoate and propanoate metabolism in iCD (Figure 4), suggesting a potential decrease in SCFA production by the microbiome, possibly due to the observed decrease in Roseburia and Faecalibacterium.

We saw an increase in glutathione transport gene abundance in UC and CD (Figure 5) and an increase in glutathione metabolism gene abundance in UC. Glutathione is a tripeptide of cysteine and glutamate, synthesized by Proteobacteria and a few streptococci and enterococci 75, which allows bacteria to maintain homeostasis during oxidative or acid stress. Inflammatory cascades include production of highly reactive oxygen and nitrogen metabolites, which are greatly increased in active IBD 76. Lamina propia monocytes also release homocysteine during inflammation, which further contributes to oxidative stress; IBD is associated with higher levels of both mucosal and serum homocysteine 77. Thus, the increases in sulfate transport, cysteine metabolism, and glutathione metabolism may reflect a mechanism by which the gut microbiome addresses the oxidative stress caused by inflammation.

CD with ileal involvement exhibited specific dysfunction at the module level. It was associated with an increase in several modules involved in glycolysis and carbohydrate transport and metabolism (Figure 5). Conversely, iCD exhibited lower abundance of genes involved in lipid metabolism and catabolism, confirming a major imbalance in energy metabolism. We observed a global decrease in nicotinamide, purine, and pyrimidine nucleotide biosynthesis modules in iCD, CD, and UC (Figure 5).

There was a decrease in vitamin biosynthesis associated with iCD, but increases in thiamine and particularly riboflavin metabolism modules (Figure 4). Interestingly, this pathway is fed by the pentose phosphate pathway, which was also overrepresented in iCD. Riboflavin is necessary for regenerating oxidized glutathione back to its reduced form, and is thus essential for pH and oxidative stress homeostasis, as is NADPH, a product of the pentose phosphate pathway. Metabolism of the sulfur-containing amino acids cysteine and methionine was increased in iCD, in marked contrast to the IBD-associated decreases in the non-sulfur-containing amino acids such as lysine and glutamine. As homocysteine is easily convertible to methionine, this may indicate a further mechanism of maintaining redox homeostasis. Alternatively, this may be connected to the iCD-specific increase in carbohydrate metabolism, as cysteine may be metabolized to pyruvate.

Finally, genes involved in pathogenesis processes, such as secretion systems and adherence/invasion, were overrepresented in iCD (Figure 4). For example, genes involved in the shigellosis pathway were more abundant in CD, and type II secretion genes were more abundant in iCD. Type II secretion is involved in the secretion of cell wall-degrading enzymes 78 and the secretion of toxins such as heat-labile enterotoxin, similar to cholera toxin 79. These functions are typical of pathobiont adherent-invasive E. coli, which have been observed to increase in iCD in our own study and others 655. This may be associated with tissue damage, either primarily as a result of toxin secretion, or secondarily as a result of stimulated cytokine production. This tissue destruction is a likely source of metabolites for microbial overgrowth, selecting for auxotrophic specialists able to thrive in this environment and resulting in the microbiome-wide loss of basic biosynthetic processes (Figures 4 and 5). This would in turn lead to further tissue breakdown, bacterial overgrowth, and community structural and functional dysbiosis.

The Ocean State Crohn's and Colitis Area Registry (OSCCAR) is a state-based, prospective inception cohort of IBD patients that was designed to study the epidemiology of IBD, to determine the incidence of IBD in Rhode Island, and to extrapolate these rates to the general population of the United States. The diverse population of over 1 million, limited geographic range, and well-circumscribed gastroenterology community of Rhode Island were ideal circumstances for establishing a prospective inception cohort of IBD patients. All but one of the 98 gastroenterologists/colorectal surgeons in Rhode Island agreed to refer patients to OSCCAR, and 11 gastroenterologists practicing in Massachusetts just over the Rhode Island border also agreed to refer their newly diagnosed IBD patients who resided in Rhode Island. Enrollment began 1 January 2008. All Rhode Island residents with a newly confirmed diagnosis of CD, UC, or indeterminate colitis were eligible for inclusion (within 12 months from diagnosis). Ethnic background of the subjects was not available for consideration in the analysis, and indeterminate colitis patients were analyzed only for other metadata and not for IBD diagnosis. Diagnosis of CD, UC, or indeterminate colitis was made by endoscopic, pathologic, or radiographic findings according to the criteria of the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) IBD Genetics Consortium. OSCCAR research protocols were reviewed and approved by three institutional review boards (Lifespan (#0214-07), the Partners Human Research Committee (#2007-P-001705), and the Program for the Protection of Human Subjects/Mount Sinai School of Medicine (#11-01479)), and all experiments adhered to the regulations of these review boards. Informed consent and HIPAA (Health Insurance Portability and Accountability Act) authorization were obtained from each subject prior to study participation. Individuals diagnosed with IBD prior to the study start date, pregnant women, those unwilling to provide informed consent for study participation, and those who were prisoners at the time of diagnosis were not permitted to enroll.

The Prospective Registry in IBD Study at MGH (PRISM) is a referral center-based, prospective cohort of IBD patients. Enrollment began 1 January 2005. Patients aged 18 years and older with a diagnosis of CD or UC based upon standard endoscopic, radiographic, and histologic criteria were eligible to participate. Controls consisted of healthy patients aged 18 years and older, from whom biopsies were obtained during colonoscopies performed for screening purposes.

Patients were excluded from the healthy volunteer group for current acute illness, if awaiting transplant, or if chronically ill (for example, renal failure, diabetes, congestive heart failure). During routine colonoscopies, subjects were offered the opportunity to donate biopsy samples. After sampling, intestinal biopsies were stored in 5% glycerol at -80°C until DNA extraction. Stool samples were kept at 4°C for less than 24 h before storage at -80°C until DNA extraction. PRISM research protocols were reviewed and approved by the Partners Human Research Committee (#2004-P-001067), and all experiments adhered to the regulations of this review board.

DNA from stool and biopsy samples was extracted using the QIAamp DNA Stool Mini Kit (Qiagen, Inc., Valencia, CA, USA) according to manufacturer's instructions and as described previously 87. The manufacturer's protocol was altered to accommodate larger stool volumes and to improve homogenization using bead-beating at several steps: a) a minimum of 2 ml of Buffer ASL and 300 mg of stool was used in the protocol; b) a ratio of 700 μl of Buffer ASL per 100 mg of stool weight was used for larger volumes using no more than 1,500 mg of stool and 10.5 ml of Buffer ASL; c) following the addition of Buffer ASL to each sample (step 2), 0.70 mm Garnet Beads (MO BIO Laboratories, Inc., Carlsbad, CA, USA) were added to the suspension and vortexed for 10 seconds; d) a second bead-beating was performed following the heating of the suspension (step 3) in 0.1 mm Glass Bead Tubes (MO BIO Laboratories, Inc.), and vortexed for 10 minutes.

The 16S gene dataset consists of 454 FLX Titanium sequences spanning the V3 to V5 variable regions. Detailed protocols used for 16S amplification and sequencing are available on the Human Microbiome Project Data Analysis and Coordination Center website 88. In brief, genomic DNA was subjected to 16S amplifications using primers designed incorporating the FLX Titanium adapters and a sample barcode sequence, allowing directional sequencing covering variable regions V5 to partial V3 (primers: 357F 5'-CCTACGGGAGGCAGCAG-3' and 926R 5' CCGTCAATTCMTTTRAGT-3'). PCR mixtures (25 μl) contained 10 ng of template, 1× Easy A reaction buffer (Stratagene, La Jolla, CA, USA), 200 mM of each dNTP (Stratagene), 200 nM of each primer, and 1.25 U AccuPrime hifi cloning enzyme (Invitrogen, Carlsbad, CA, USA). The cycling conditions for the V3-V5 consisted of an initial denaturation of 95°C for 2 minutes, followed by 25 cycles of denaturation at 95°C for 40 s, annealing at 50°C for 30 s, extension at 72°C for 5 minutes and a final extension at 72°C for 7 minutes. Amplicons were confirmed on 1.2% Flash Gels (Lonza, Rockland, ME, USA), purified with AMPure XP DNA purification beads (Beckman Coulter, Danvers, MA, USA) according to the manufacturer, and eluted in 25 μl of 1× low TE buffer (pH 8.0). Amplicons were quantified on Agilent Bioanalyzer 2100 DNA 1000 chips (Agilent Technologies, Santa Clara, CA, USA) and pooled in equimolar concentration. Emulsion PCR and sequencing were performed according to the manufacturer's specifications.

Sequences were processed in a data curation pipeline implemented in MOTHUR 89, which removed sequences from the analysis if they were less than 200 nucleotides or greater than 600 nucleotides, had a low read quality score (< 25), contained ambiguous characters, had a non-exact barcode match, or had more than 4 mismatches to the reverse primer sequences (926R). Remaining sequences were assigned to samples based on barcode matches, and barcode and primer sequences were then trimmed. Chimeric sequences were identified using the ChimeraSlayer 90 algorithm, and reads were classified with the MSU RDP classifier v2.2 91 using the taxonomy maintained at the Ribosomal Database Project (RDP 10 database, version 6). Sequencing depth after processing averaged 2,860 (standard deviation 1,730) reads per sample.

To construct an approximate gene catalog for each sample community, we used the gene content of 1,119 KEGG reference genomes to infer the approximate gene content of our detected phylotypes. We first matched the FastTree GreenGenes (GG) phylogeny 92 annotated with these KEGG genomes' organisms against the RDP taxonomy used for phylotyping. Each clade in the RDP taxonomy was mapped to the clade within the GG phylogeny that maximized the Jaccard index of overlapping named descendant genomes. That is, each genus-level phylotype was assigned to the GG clade containing the most genomes from that genus and fewest from other genera. Higher-level clades continued this pattern using the Jaccard index as an optimality criterion. The gene contents for ancestral clades were then reconstructed across the GG tree, beginning with each reference genome (tree leaf) summarized as a vector of KEGG ortholog (KO) 70 copy numbers (0, 1, or multiple copies of the gene annotated within the genome). Gene contents of each parent GG clade g were calculated by averaging all descendant genomes' h KO vectors, with weight w(g, h) inversely exponential to phylogenetic distance:

g ⇀ = ∑ h ∈ d e s c e n d a n t s ( g ) w ( g , h ) h ⇀ / ∑ h ∈ d e s c e n d a n t s ( g ) w ( g , h )

w ( g , h ) = 2 = d i s t ( g , h )

for GG tree nodes g and h separated by phylogenetic branch length dist(g, h) and annotated with KO genome vectors g ⇀ and h ⇀ .

Using this vector representation of genomes, the abundance of an individual gene family (KO) i in a community due to the presence of a specific phylotype g is the product of the corresponding gene count g ⇀ [ i ] and the measured abundance of phylotype g. Therefore, the total relative abundance of each KO was estimated for each sample by adding the individual contributions of all phylotypes present in the sample. Using this method, we inferred the functional composition for each sampled community. The inference process's accuracy was validated by comparing inferred KO abundances in 16S datasets from the Human Microbiome Project with their metagenomically sequenced counterparts (Additional file 15).

Inferred per-community gene (KO) abundances were subsequently reconstructed into microbial pathway relative abundances using HUMAnN, the Human Microbiome Project metabolic reconstruction pipeline 71. KOs were grouped into pathways represented as gene sets using HUMAnN, which chooses pathways by maximum parsimony using MinPath 93 and computes each pathway's relative abundance as a smoothed average over all genes within it, taking into account outliers and gap filling. We ran HUMAnN three times to reconstruct three complementary types of pathways from these genes: small metabolic modules (using KEGG's conjunctive normal form logic), large metabolic pathways, and Gene Ontology terms (using annotation-to-KO mappings from nine well-characterized KEGG microbes: ban, cje, cpe, eco, nse, pae, sce, son, and vch). For each of these three types of pathway, HUMAnN input the inferred relative abundances of all genes in each sample, and output the relative abundances of pathways within the sample. Subsequent analysis handled these sample-by-pathway relative abundances in the same manner as sample-by-clade microbial abundances.

Inverse Simpson diversity, Chao1 richness (using the R fossil package), and Pielou evenness were calculated for clade abundance, KEGG pathway and module abundance, and Gene Ontology term abundance 94959697. Next, data were pre-processed for quality control before modeling. Clinical metadata were removed when more than 10% of data were missing, or when they did not vary in value over the available samples. Clades, pathways, and features of very low abundance (< 0.001 in ≥ 90% of samples) and feature outliers outside of the lower or upper outer fence (3× interquartile range) were removed. Missing data were imputed for significance testing with the mean abundance of the sample; missing factor metadata were imputed with a 'NA' factor level using the na.gam.replace function from the R package 98. Unless stated otherwise, all subsequent analyses and calculations were performed using these processed data. After processing, 228 and 231 samples passed quality control for clade abundance and functional abundance analyses, respectively.

Finally, clades and functions were tested for statistically significant associations with clinical metadata of interest by using a novel multivariate algorithm. Each clade (excluding ecological measures) was normalized with a variance-stabilizing arcsine square-root transformation and evaluated with a general linear model (in R using the glm package). Model selection for sparse data was performed per clade using boosting (gbm package 99). A multivariate linear model associating all available metadata with each clade independently was boosted, and any metadata selected in at least 1% of these iterations was finally tested for significance in a standard generalized linear model. This composite model was thus of the form:

arcsin ( y i ) ) = β 0 + ∑ p β p X i , p + ε i , i = 1 , . . . , n

where p are the clinical metadata selected from boosting.

Within each metadatum/clade association independently, multiple comparisons over factor levels were adjusted using a Bonferonni correction; multiple hypothesis tests over all clades and metadata were adjusted to produce a final Benjamini and Hochberg false discovery rate 100. Unless otherwise indicated, significant association was considered below a q-value threshold of 0.25; the KEGG pathway sulfur metabolism (ko00920) had an average q-value of 0.26 for association with Crohn's disease. Multiple factor analysis was performed to visualize the relationships within heterogeneous factor data as well as with a select group of taxa found to be significantly associated with metadata (using the FactoMineR R package 101). Total abundances and significant associations between metadata, taxa, and functions are listed in Additional files 1 and 11.

To determine whether SFB were present in samples, three sequences of SFB (X80834, X87244, and X77814) from three species (chicken, rat, and mouse) were aligned by blastn, using both a 20 and 15 seed word. No sequences were found with ≥ 95% identity over an alignment length of at least 100 nucleotides. The average sequence length from the study was 435 nucleotides.

To provide internal validation of inferred microbial community gene and pathway compositions, stool DNA from seven healthy controls and four CD patients was subjected to metagenomic shotgun sequencing. Libraries were constructed with the Illumina Nextera XT kit and sequenced on an Illumina MiSeq using 2 × 150 bp paired-end sequencing according to the manufacturer's instructions. This resulted in sequencing depths ranging from 3.9 to 270 meganucleotides, average 119 meganucleotides, from which microbial community function was determined with HUMAnN 71 as described above.

Sequences generated in this study are publicly available (NCBI BioProject ID numbers 82111 and 175224).