Multiplex IF assays are highly valuable in generating data that enables a deeper understanding of cell phenotypes, their functional status, and spatial relationships within tissues across different diseases or therapeutic interventions. However, the staining of targets is not always consistent between tissue samples and can impact the quality and/or efficiency of data generation.

In this article, we give our perspective on how Artificial Intelligence (AI) Deep Learning can be utilized to address staining variance across Multiplex IF image sets.

In our 11 years of experience specialising in image analysis, we’ve noted histology multiplex immunofluorescence (mIF) assays being used with increasing regularity in clinical R&D studies, especially in immuno-oncology, where multiple cell types are involved in the pathophysiology of various cancers. Determining the roles and interactions of the different cell types, both at the spatial and functional level, can provide valuable insights into the therapeutic mechanism of action, patient response and ultimately support more informed decision making along the R&D process.

The application of Deep Learning techniques in Digital Pathology is commonly used to support more efficient or accurate identification of tissue morphology, including segmentation of tissue regions of interest (ROI) or detection of various cell types. The value of these applications is already positively impacting both R&D and diagnostic healthcare workflows.

We work on a lot of clinical multiplex IF R&D studies where we analyse images stained with various multiplex assay technologies, from 4-plex up to 40-plex markers. Here, Deep Learning techniques can support more accurate identification of tissue ROI and cell types, especially in Oncology studies containing different tumor types. However, other image analysis challenges exist, especially around how to deal with staining variance for each target within the multiplex assay across either large study cohorts containing different cancer types, or where technical differences arise between batch staining runs.  

In our recent webinar, we highlighted how we developed an algorithm using Deep Learning within Visiopharm software to recognise CD3 positive cells that formed part of an 8-plex mIF panel applied to a multi-cancer TMA. Across the different cores, we noted that CD3 positive cells presented with different stain intensity levels and that traditional non-Deep Learning approaches would have required multiple intensity thresholds (and therefore algorithms) to accurately separate positive from negative CD3 cells across the TMA.

Examples of staining variance observed across cores within a multi-cancer TMA

Note: Click the image below to open a higher resolution version (opens in a new tab).

• Panel 1: Examples of 4 cores showing CD3 (red), Cytokeratin (cyan), and DAPI/nuclei (blue)
• Panel 2: CD3 channel only per core showing a range of weak and intensely staining CD3 cells, also some areas of non-specific staining (collagen / structural proteins).

By training the Deep Learning App to recognise weak and intensely stained CD3 cells, as well as recognising weak and intense non-specific staining in the CD3 channel, a Deep Learning output feature was created by the neural network that consistently picked out true CD3 cells across the various staining scenarios presented.

 The created Deep Learning output feature acted as the basis for us to perform further algorithm post-processing steps to accurately create CD3 positive and negative cells per core. This approach reduced the time taken for us to accurately quantify CD3 cell detection across the cores, while also improving the quality of cells detected, especially in regions of higher non-specific signal.

Algorithm steps to train a Deep Learning network to recognise variant CD3 staining

Note: Click the image below to open a higher resolution version (opens in a new tab).

• Panel 1: Magnified areas within each core showing examples of CD3 (red) cell numbers and stain intensity variance in individual cells across different cores. Nuclei showing in blue (DAPI).
• Panel 2: CD3 channel showing a range of weak and intensely staining CD3 cells, also some areas of non-specific staining (collagen / structural proteins).
• Panel 3: Examples of the annotated cells including CD3 positive (red) and CD3 negative (blue) used as algorithm training labels.
• Panel 4: Examples of the CD3 Deep Learning output feature created from the trained algorithm. Feature for CD3 positive cells shown in white.
• Panel 5: Examples of the formed CD3 positive (red outlines) and negative (blue outlines) cells created using further post-processing steps within the algorithm.

In summary

Deep Learning provides a valuable tool during algorithm development to help manage stain variance across sample sets stained with high plex assays. Ideally, stain variance should be minimized during steps preceding image analysis, but this is not always straightforward to identify and eradicate.

Using Deep Learning also requires a mind shift from traditional thinking, where it has been important to define a single intensity threshold per marker that is standardized for a specific study. Although this is important, especially where a biomarker intensity profile is the primary read-out across different samples (i.e., H-score), it can become impractical to adhere to for multiplex assays applied to large sample sets. Utilising Deep Learning enables an adaptive thresholding approach, which can take into account target specific, or off-target, stain variance on a section by section basis.

Our experiences to date indicate that Deep Learning approaches can improve the accuracy of detected cell phenotypes across variant staining conditions, thereby enhancing the application and value of these crucial assays within clinical R&D.

Associated  Links

• Watch the full webinar
• Arrange a chat with our experts to discuss your study
• Connect with Lorcan on LinkedIn