Most experimental cell populations are contaminated by a small amount of debris, cell conjugates, buffer precipitates, and air bubbles. The design of FACS instruments usually does not allow perfect discrimination of these contaminants from single, living cells during data acquisition, and hence they can end up in the raw data. To a certain extent we can discriminate contaminants from living cells using the morphologic properties provided by the FSC and SSC parameters. The joint distribution of FSC and SSC for transformed mammalian cells typically exhibits an elliptical shape, and most contaminants separate clearly from this main population (Figure 1a). The core distribution of healthy cells is approximated by a bivariate normal distribution in the (FSC, SSC) space, allowing the identification of outliers by their low probability density in that distribution. Thus, measured events that lie outside a certain density threshold can be regarded as contamination. We fit the bivariate normal distribution to the data by robust estimation of its center and its 2 × 2 covariance matrix (Figure 1b). This is appropriate if the cell population is homogeneous, the proportion of contaminants is small, and the phenotype of interest is not itself associated with large changes in the FSC or SSC signal. A rough pre-selection using some fixed FSC and SSC threshold values, as provided by most FACS instruments, further increases robustness.

To see how this affects the data, Figure 1 panels c and d show scatterplots of the two fluorescence channels measuring the perturbation and the phenotype before and after removal of contaminants. We observe a reduction in the proportion of data points with very small fluorescence values in both channels after removing contaminants. This is reasonable because the fluorescence staining is intracellular, and hence cell debris is not expected to emit strong fluorescence. In addition, we have removed some of the data points with very high fluorescence levels, which apparently correspond to cell conjugates.

For our example data it is possible to determine global, experiment-wide parameters of the core distribution of healthy and well measured cells. However, some experimental settings may also demand adaptive estimates, for example if the cell morphology is expected to change as a result of the perturbation (as is the case for apoptotic cells) or if systematic shifts occur during the course of one experiment.

Regardless of the presence of fluorochromes, every cell emits light when it is excited by a laser - a phenomenon referred to as autofluorescence. Autofluorescence intensities frequently correlate with cell size, and through this effect often spurious correlations between different fluorescence channels can occur. In our data, the unspecific autofluorescence adds both to the specific fluorescence emitted by the fluorochrome-conjugated antibody measuring the phenotype and to that of the YFP-expressing construct, and it is positively correlated with cell size (Figure 2a,b). This results in an apparent, unspecific increase in the response variable for higher levels of perturbation (Figure 2c). To recover the specific signal we use FSC as a proxy for size, and fit the linear model:

x_total = α + βs + β_specific   (1)

Where x_total is the measured fluorescence intensity, s is the cell size as measured by the forward light scatter, α and β are the coefficients of the model, and x_specific is the specific fluorescence. We compute α and β by robust fit of a linear regression of x_total on s, and obtain estimates for x_specific from the residuals (Figure 2d). This is done for each fluorescence channel individually. The artifactual correlation due to autofluorescence is absorbed by β. The parameter α absorbs baseline fluorescence, as discussed below.

In our data we often observe variation in the overall signal intensities for different wells on a microtiter plate (Figure 3a), which may be due to various drifts in the equipment, such as changes in laser power or pipetting efficiencies. Although such effects should ideally be avoided, and large variations should prompt reassessment of the experimental setup, small variations are adjusted by the model described by equation 1. In particular, they are fitted by the intercept term α. The biologically relevant information is retained in the residuals. A common baseline of the adjusted values is obtained by adding the mean of α averaged over all wells (Figure 3b).

A natural approach to modeling binary responses is to dissect the data into four subtypes: perturbed versus nonperturbed cells, and cells exhibiting the effect of interest versus nonresponding cells (Figure 5a). Thresholds for this separation can be obtained either adaptively, for each well, or more globally, for the whole plate. Because of the potential problems with over-fitting in the adaptive approach, we choose the latter, making use of the premise that the values of the pre-processed data are comparable across the plate. Figure 5b shows thresholds determined from a high percentile (99%) of the data from a negative control.

An estimator for the odds ratio, a measure of the effect size, is defined by the following equation:

O R = p p + 1 p n + 1 ⋅ n n + 1 n p + 1       ( 2 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8tuQ8FMI8Gi=hEeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciGacaGaaeqabaqadeqadaaakeaacaWGpbGaamOuaiabg2da9maalaaabaGaamiCaiaadchacqGHRaWkcaaIXaaabaGaamiCaiaad6gacqGHRaWkcaaIXaaaaiabgwSixpaalaaabaGaamOBaiaad6gacqGHRaWkcaaIXaaabaGaamOBaiaadchacqGHRaWkcaaIXaaaaiaaxMaacaWLjaWaaeWaceaacaaIYaaacaGLOaGaayzkaaaaaa@49E7@

The symbols on the right hand side of equation 2 are defined in Figure 5a. Pseudo-counts of 1 are added in order to avoid infinite values in the case of empty quadrants 22. It is often convenient to consider the logarithm of the odds ratio, because it is symmetric for upward and downward effects. To test for the significance against the null hypothesis of no effect, we use the Fisher test 23.

Sample results from a screen aiming to identify activators of the apoptosis pathway are shown in Figure 6. Overexpression of the Fas receptor protein in Figure 6b leads to strong activation of apoptosis, as indicated by both high effect size and a significant P value. This is consistent with the cellular role played by the Fas receptor, which mediates apoptosis activation as a consequence of extracellular signaling. Overexpression of the YFP protein in Figure 6a apparently does not affect apoptosis, proving that the activation in Figure 6b is not caused by the fluorescence tag alone.

The gradual nature of these types of responses supports the use of regression analysis. Because the effect may deviate from linearity in the range of perturbations that we observe, we use a robust local regression fit:

y = m(x) + ε   (3)

Where x is the perturbation signal, y is the response, m is a smooth function (for example, a piece-wise polynomial), and ε is a noise term. We obtain an estimate of m from the function locfit.robust in the R package locfit 24. This also calculates

δ = m^(x0) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8tuQ8FMI8Gi=hEeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciGacaGaaeqabaqadeqadaaakeaaceWGTbGbaKaadaqadiqaaiaadIhadaWgaaWcbaGaaGimaaqabaaakiaawIcacaGLPaaaaaa@37A7@   (4)

which is a robust estimate of the slope of m at the point x₀. x₀ is an assay-wide, user-defined parameter that corresponds to a mild perturbation that does not deviate strongly from the physiologic value. This approach is resistant to nonlinear, biologically artifactual effects caused by perturbations that are too strong, without the need for a sharp cut-off. To obtain a dimensionless measure of effect size, we divide

z = δ δ 0       ( 5 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8tuQ8FMI8Gi=hEeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciGacaGaaeqabaqadeqadaaakeaacaWG6bGaeyypa0ZaaSaaaeaaiiGacqWF0oazaeaacqWF0oazdaWgaaWcbaGaaGimaaqabaaaaOGaaCzcaiaaxMaadaqadiqaaiaaiwdaaiaawIcacaGLPaaaaaa@3D0C@

Where δ₀ is a scale parameter of the overall, assay-wide distribution of δ. We use the median absolute value of all δ in the assay. A simple measure of the significance against the null hypothesis of no effect is obtained through dividing the estimate m^(x0) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2Caerbhv2BYDwAHbqedmvETj2BSbqee0evGueE0jxyaibaiKI8=vI8tuQ8FMI8Gi=hEeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciGacaGaaeqabaqadeqadaaakeaaceWGTbGbaKaadaqadiqaaiaadIhadaWgaaWcbaGaaGimaaqabaaakiaawIcacaGLPaaaaaa@37A7@ by its estimated standard deviation, and by assumption of normality a P value is obtained.

The plots in Figure 7 show the fitted local regression for three examples from a cell-based assay targeting the MAPK pathway. As a result of the overexpression of the phospholipase C δ4 (PLCD4) protein, our method detects a significant induction of extracellular signal-regulated kinase (ERK) activation (Figure 7a) - a finding that is consistent with previous reports 25. As expected, overexpression of the dual specificity protein phosphatase (DUSP)10 protein strongly inactivates MAPK signaling (Figure 7b), whereas overexpression of the YFP protein has no effect (Figure 7c).

The P values obtained from the previous section test the statistical association between the fluorescence signals from the overexpressed YFP-tagged proteins and the reporter-specific antibodies for the cell population in one particular well. It is important to note that this only takes into account the cell-to-cell variability within that well and does not reflect higher levels of experimental and biologic variability. Hence, the results from a single well cannot simply be taken as a measure of biologic significance. To gain confidence in the biologic significance of a result, the next step is to consider measurements over several independently replicated wells.

The most obvious approach to summarizing data from replicate measurements for the same gene is to combine the effect size estimates and the P values from the individual replicates using tools from statistical meta-analysis 26. However, because all of the data are available, the more direct and probably more efficient approach is to generalize the previous analysis methods and to deal with replicate wells. In particular, for stratified contingency tables in the case of binary responses, we use the stratified Χ²-statistic in the Cochran-Mantel-Haenszel test 27. For stratified continuous responses we extend equation 3:

y = y_i + m(x - x_i) + ε   (6)

Where i = 1, 2, ... counts over the replicates and x_i and y_i are replicate specific offsets. Again, in both cases we obtain estimates of effect size as well as significance.

Because of the large number of tests performed, it is necessary to adjust for multiple testing. Good software for this is available in the R packages qvalue and multtest, and we recommend the reports by Storey 28 and Pollard 29 and their coworkers for methodologic background.

Even after multiple testing adjustment, one will often encounter situations in which for many of the screened genes the null hypothesis of no effect will be rejected, although the effect sizes (equations 2 and 5) may be quite small for most of them. This can happen because of the large number of cells observed for each gene, and it is a well known phenomenon of statistical testing; when the number of data points becomes large, hypothesis tests will eventually reject any null hypothesis that differs from the truth, even in the most negligible manner 30. Such cases are unlikely to be biologically interesting. Hence, for biologically relevant effectors we require both the effect size estimate to be above a certain threshold and the adjusted P value to be small.

Finally, as with any biologic assay, to corroborate conclusively the role of a protein in the cellular process of interest, independent validation experiments must be conducted according to best experimental practice.

A simple but useful way to visualize bivariate data is by means of a scatterplot. However, it is difficult to get a good impression of the distribution of the data when the number of observations is large and the points become too dense (Figure 8a). This is a problem for cytometry data with often more than 20,000 data points. A way to circumvent this limitation (which has already been applied in some of the previous figures) is by plotting the densities of the data points at a given region 31 instead of individual points (Figure 8d) or, alternatively, by plotting each single point using a color coding that represents the density at its position (Figure 8c). We prefer false color coding to the commonly used contour plots (Figure 8b) because we find it more intuitive. By further augmenting false color density plots with outlying points, one can also visualize the data in sparse regions of the plot. We compute densities using a kernel density estimate.

Most high-throughput applications in cell biology are carried out on microtiter plates which come in different formats, usually as a rectangular arrangement of 24, 96, 384, or 1536 wells. Each well may contain cells that have been treated in a different manner. An intuitive approach for visualization is to use the familiar spatial layout of the plate. Figure 9a shows an example of what we call a plate plot for a 96-well plate. It indicates the number of cells identified in each well. The consistently low number of cells on the edges of the plate suggests a handling problem, and subsequent analysis steps are possibly affected by this artifact. Other quantities of interest often include the average fluorescence of each well, for example to monitor expression efficiency or to detect artifactual shifts in the response.

Plate plots can also be used to present qualitative variables. Figure 9b shows the negative log transformed odds ratios from the statistical analysis of a 96-well plate from a cell proliferation assay. Negative values indicate inhibition of cell proliferation and are colored in blue, whereas positive values correspond to activation as indicated in red. The attention of the experimenter is immediately drawn to the few interesting wells and spatial regularities are easily spotted. In this example, we can compare the upper and lower halves of the plate; the top half contains cells transfected with carboxyl-terminally tagged constructs and the bottom half contains cell transfected with amino-terminally tagged constructs of the same genes. Additional information is added to the plot by using further formatting options, for instance crossing out of wells discarded from analysis or plotting additional symbols on wells with controls.

The amount of information included in a plate plot can be extended further by decorating it with tool tips and hyperlinks. When viewed in a browser, a tool tip is a short textual annotation, for example a gene name, that is displayed when the mouse pointer moves over a plot element. A hyperlink can be used to display more detailed information, even a graphic, in another browser window or frame. For example, underlying each value that is displayed in a plate plot such as Figure 9b is a complex statistical analysis, the details of which can be displayed on demand by hyperlinking them to the corresponding well icons in the plate plot. The reader is directed to the online complement 32 for an interactive example. Using plate plots in this way provides a powerful organizational structure for drill-down facilities because potentially interesting candidates are easily identified on a plate and the range of detailed information enables the experimenter to audit steps of the analysis procedure.

Because experiments are done in replicates, another level of visualization is needed to compare multiple measurements of the same gene over several plates. For a limited number of replicates the plate plot concept can be utilized. Besides colored circles, as in Figure 9 panels a and b, its implementation allows us to plot arbitrary graphs at each well position. In Figure 9c we use segmented charts to display the results from four replicate experiments (we call this a 'pizza plot'). For more extensive datasets, Figure 10 shows how hyperlinked box plots can be used to display multiple relevant aspects of the data. In this example they allow exploration of the effect of the orientation of the carboxyl-terminal or amino-terminal YFP fusion in the expression vectors.