Thus, the activity of many of the compounds are unknown and no loss signal backpropagated from those neurons. The distribution of compounds between different splits was done based on structural similarity. The model and hyperparameter selection were done using only the training and validation splits, while the test set was only used to report final performance.
As it has been shown that brightfield images can be used to predict Cell Painting features15, we wanted to investigate if the information content in the images would be sufficient to predict bioactivity of compounds. To further strengthen the practicality of our approach, we analyze prediction performance and robustness across various assay types, technologies, and target classes to identify specific targets and assays that are particularly well-suited for bioactivity prediction. Furthermore, we explore different input modalities for bioactivity prediction, including fluorescence images, brightfield images, and image features extracted from the fluorescence images using classical image analysis approaches. We employ a large-scale general purpose Cell Painting screen to capture phenotypic profiles of a library of available compounds and train a model using small, focused bioactivity assay readouts for specific targets. So-called structure activity relation (SAR) models are a family of computational methods, used to make bioactivity predictions or property predictions i.e., using computational methods and models to estimate bioactivity or properties of chemical compounds. Top ranked compounds in four of the assays were selected for follow-up validation in secondary screening.
Multi-Layer Perceptrons (MLPs) were used for the cell-feature-based model and the structure-based, described below in section ‘Cell-feature model’ and section ‘Structure fingerprints model’ For the Fluorescence and Brightfield images ResNet50s were used. Following this, only assays with enough activity data among the remaining compounds were kept (more than 50 active and inactive compounds). Overall, the results paint a positive picture for phenotypic-based bioactivity models to complement structure-based predictors, where data can be generated in a cost-effective manner and with several use-cases. Notably, kinase targets and cell-based assays exhibited strong performance, and additional trends can be recognized albeit at small sample size (Fig. 3).
Prediction setup
A hyper-parameter search was performed using nested cross-validation in each one of the cross-validation splits, using three splits for training, one for nested validation and one for nested testing. All models were trained on two NVIDIA-Tesla 32 Gb GPUs, using pytorch DDP38. Area Under the Receiver Operating Characteristic Curve (ROC-AUC) was used to evaluate the model’s ability to separate the actives from inactive. The models were trained with Binary Cross-Entropy combined with Focal Loss37.
A Box plot of each modality type’s average ROC-AUC computed over each assay. To this end, we extracted hand-crafted image features, hereafter referred to as Cell-Features, using the Columbus image-analysis software. Brightfield imaging has some advantages compared to Cell Painting-stained cells as it can be performed on live cells and does not require staining of the cells and can be performed on simpler microscopes.
How to Succeed in High Throughput Screening and Hit Identification
- We demonstrate efficient learning despite the inherent noise in single-point activity data.
- All models were trained on two NVIDIA-Tesla 32 Gb GPUs, using pytorch DDP38.
- The increased performance on kinase targets could be attributed to the known promiscuity of kinase inhibitors, which can affect multiple cellular pathways, leading to stronger phenotypic responses that are more readily identified by the model.
- The model and hyperparameter selection were done using only the training and validation splits, while the test set was only used to report final performance.
- Thus, there is an interest in using as biologically relevant assays as possible early in the screening cascade.
This shows that the information captured in brightfield images can be linked to bioactivity in a wide range of assays and targets, which may justify using brightfield images in some cases despite their slightly inferior performance. The fluorescence and brightfield images were used to train ResNet50 models, while the cell-features and structure-based data were used to train multi-layer perceptrons (See Materials and Methods for details). The performance difference between structure- and image-based model is significant for cell-based assays but does not reach significance for biochemical assays. While structure-based bioactivity prediction is attractive as it requires no in vitro data, alternative input representation can avoid the problems SAR models have with scaffold hopping and increasing the diversity amongst predicted hits.
Applying a post-hoc Nemenyi’s test, we find that the performance differences are significant between all modalities except for brightfield and structure. This approach reached an average ROC-AUC of vegas casino download 0.744 ± 0.108 compared to the cell-feature based model at 0.726 ± 0.115. These image-based modalities were then compared against a standard structure-based approach using Extended Connectivity Fingerprints17 (labeled Structure). As described above, we observed encouraging results using a multiplexed fluorescence Cell Painting screen to capture phenotypic profiles of a library of compounds. The average performance for these 29 assays was 0.660 ± 0.094 ROC-AUC. Notably, our results align closely with the performance reported for the supervised ResNet model by Hofmarcher et al. (0.731 ± 0.19 ROC-AUC)8 and the linear probing contrastive learning model (CLOOME) recently reported on the same dataset (0.714 ± 0.20 ROC-AUC)12.
We investigate the potential of deep learning on unrefined single-concentration activity readouts and Cell Painting data, to predict compound activity across 140 diverse assays. HTS uses miniaturised assays and automation to screen large compound libraries therefore generating data rapidly and cost effectively. HTS identifies potential hits which show binding or activity against a particular biological target or cellular phenotype. It involves screening thousands or millions of compounds through a previously developed biological assay using automation. Assay development is one of the most critical stages in the hit identification process as the quality of the assay and robustness of the automation infrastructure determines the quality of the data. Our capabilities include target-based approaches (e.g., biochemical, cell-based, biophysical, including ASMS and DEL technology, fragment-based screening etc), various cellular and phenotypic screening and in silico approaches.
High-Throughput Screening of Model Bacteria
These types of approaches promise to efficiently enrich likely hit compounds into focused compound sets. One strategy to accelerate hit finding is to use computational methods to prioritize and select compounds deemed more likely to be active. Thus, there is an interest in using as biologically relevant assays as possible early in the screening cascade. This approach has the potential to reduce the size of screening campaigns, saving time and resources, and enabling primary screening with more complex assays. Identifying active compounds for a target is a time- and resource-intensive task in early drug discovery. What is a cell-based HT screening approach?
Introduction: Cell-Based Assays for High-Throughput Screening
Depending on the input modality, different Machine Learning models were used. Using RDKit Butina ClusterData function, the ECFP4 representations was used to group the data into unique clusters. Using RDKit36, all compound SMILES representations were converted to ECFP Bit.
Table of contents (14 protocols)
For this matter, screening chemical libraries and monitoring bacterial growth responses to find potential growth inhibitors is of interest in many pharmaceutical and biotechnological companies . Emphasis is made on recent CRISPR/dCas-based screens. We devote some emphasis to high-content screening, which is becoming very popular. Binding-based methods are also surveyed, including NMR, SPR, mass spectrometry, and DSF.
Mature Content
- In terms of assay technologies (Fig. 3f), predictions were especially accurate in chemiluminescence assays, and less so in assays using spectrophotometry.
- Each dot represents the average Tanimoto similarity score per assay over all cross-validation splits.
- Our approach has the potential to reduce the number of compounds screened, as well as the number of assays and experiments required in drug screening cascades, which in turn could allow for early screening of focused compound sets in assays of higher biological relevance.
- Before proceeding with large scale screening, pilot studies are recommended to validate the performance of the assay.
Evaluation on the held-out test sets revealed that the predictive performance of the whole image fluorescence-based approach outperformed all other approaches (Fig. 2a). These factors could significantly reduce the cost of the assay and enable kinetic assays, although potentially at the expense of less informative image data. 62% of the assays achieved an ROC-AUC of 0.7 or more, which we deem good performance, while 30% reached 0.8 or higher (indicating very good performance), and a further 7% reached 0.9 or higher (indicating excellent performance). It was then trained to predict bioactivity readouts for each of the 140 assays. The model was pretrained using ImageNet11 and modified to accept 5-channel fluorescence images as input.
Because the initial screening assays are often very simple representations of the target biology, they run the risk of producing false positive and negative results. Because of this, hit finding is generally done with simple assays such as biochemical assays to enrich the compound set before more resource-intense assays can be used further down the cascade. Accurate bioactivity prediction using morphological profiles could streamline the process, enabling smaller, more focused compound screens. Another important aspect of cell-based HT assays is the response of the organism of interest through the primary screen. On the other hand, cell-based assays discussed include viability, reporter gene, second messenger, and high-throughput microscopy assays.
Our image-based model on average outperforms a structure-based model. Concurrent work has shown that cell feature-based model performance is often comparable23 or slightly superior24, aligning with our observations. There has been limited work comparing models based on chemical structure and those based on imaging data22. These early studies employed binary activity data derived from dose-response curves, expressed as pXC506 or IC50/EC508 of the given compound in the given assay. Previous work by Simm et al. and Hofmarcher et al. established the information link between phenotypic screening data and assay activity6,8.
Before being sent to the network as input during training, the images were augmented, including spatial down sampling, z-normalization, random cropping, horizontal and vertical flipping, random 90-degree rotations and color shifting. The images were pre-processed and normalized such that the top and bottom 1 percentile intensity values were clipped for each image to remove noise and outliers. Fluorescent microscopy images were stored as 16-bit TIFFs of size 1992×1992. We report both the mean ROC-AUC over all assays as well as the individual ones.
Feedback loops between screening data and compound selection are key to incorporate knowledge into the next iteration. Before proceeding with large scale screening, pilot studies are recommended to validate the performance of the assay. Our compound libraries are available as single compounds but also in pools, optimised for affinity selection mass spectrometry screening (ASMS). With an excellent combination of disease biology knowledge as well as drug discovery and development expertise, Evotec selects the most suitable screening strategy for your target class and therapeutic area of interest. The enrichments at different percentiles were then calculated using bootstrapping of the activity values of the compounds above that percentile. For each follow-up assay, the top 5% of compounds deemed most likely to be active in the corresponding HTS assay, were randomly sampled.
We validated the model’s performance in follow-up experiments in secondary assays. These types of data are relatively inexpensive to produce and can therefore readily be used as a basis for training the prediction models for targets. Furthermore, the enrichment levels we observed were high enough to reduce cost and speed up the screening process by filtering in silico compounds according to the ones the model predicts to be active. In summary, the assays probed in our follow-up experiments showed that the model performance was conserved through replication, even perhaps slightly better than what was expected.
To calculate enrichment, using bootstrapping we probed the top 5% of ranked compounds for each assay (accordingly, the theoretical maximum enrichment from this experiment is 20x). While ROC-AUC helps understand the predictive performance of the model, ultimately, what we care about is how the model can be used to enrich the compound sets. Our results suggested that not only do the model predictions carry over to follow-up experiments, but that the expected range of performance is consistent as well. The ROC-AUC reported in our experiments was computed using only the randomly selected compounds to keep the values comparable with the primary assay. The majority of the top-ranked 5% of compounds were randomly sampled and included in the follow-up assay, along with selection of compounds selected uniformly at random (at least 1000 compounds in total). In each of the selected follow-up assays, a ranked list of predicted bioactivities was produced by our model.
The trained model is then used to predict the bioactivity of the compounds in the entire compound library, enabling the selection of compounds most likely to modulate the intended target (Fig. 1a). These models rely on compound structure information to make predictions of compound activity on a particular target. Drug discovery campaigns typically rely on high throughput screening (HTS) for hit finding i.e., the process of identifying and selecting chemical compounds with biological activity towards a target and the potential to be developed into a drug. In many cases, the high prediction performance can be achieved using only brightfield images instead of multichannel fluorescence images.
Recently there has been a strong interest in combining compound structure information with activity fingerprints leading to improved performance in bioactivity prediction4. ROC-AUC values for each of the four assays were calculated using the randomly sampled subset of compounds, green showing the average performance in the original HTS assays and blue representing the performance when using secondary screen activity readouts. We put the predictions of our Cell Painting-based model to the test by running secondary assays for the same targets. Previous studies that have aimed to predict bioactivity from image-based assays have limited their analysis to a single primary assay. 1b, the predictive performance of the Cell Painting fluorescence-image based model varied widely from assay to assay, ranging from 0.96 to 0.48 ROC-AUC.