Group sparsity model for image unmixing

• US 10,540,762 B2
• Filed: 08/22/2016
• Issued: 01/21/2020
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: Ventana Medical Systems Incorporated (Roche Holding AG)
• Original Assignee: Ventana Medical Systems Incorporated (Roche Holding AG)
• Inventors: Chukka, Srinivas, Chen, Ting
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US15/243,899
• Publication Number:
  US 20160358335A1
• Time in Patent Office: 1,247 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/698 Matching; Classification

Abstract

Systems and methods described herein relate, among other things, to unmixing more than three stains, while preserving the biological constraints of the biomarkers. Unlimited numbers of markers may be unmixed from a limited-channel image, such as an RGB image, without adding any mathematical complicity to the model. Known co-localization information of different biomarkers within the same tissue section enables defining fixed upper bounds for the number of stains at one pixel. A group sparsity model may be leveraged to explicitly model the fractions of stain contributions from the co-localized biomarkers into one group to yield a least squares solution within the group. A sparse solution may be obtained among the groups to ensure that only a small number of groups with a total number of stains being less than the upper bound are activated.

Claims

1. A system configured to analyze tissue obtained from a biological specimen, said system comprising:

a color data storage module to store, for each of a plurality of markers, color data indicative of a color of tissue marked by the respective marker;

a co-location data storage module to store co-location data defining a plurality of groups of said markers, each group consisting of markers having an affinity to a respective common tissue feature, wherein a tissue feature is a characteristic of a tissue that is indicative of a medical condition;

a tissue image data storage module to store a plurality of pixels representative of a tissue image, each pixel comprising color information; and

a tissue image analysis module to unmix said tissue image, wherein said tissue image analysis module is configured to read said color data from said color data storage module, said co-location data from said co-location data storage module and said pixels from said tissue image data storage module, and to calculate, for each of said pixels and for each of said groups, a linear combination of the colors of the markers of the respective group that yields a minimum difference between said color information of the respective pixel and said linear combination of colors, and wherein, for each of said pixels, said tissue image analysis module is to determine a group for which said minimum difference is smallest and outputs said tissue feature of said group as an analysis result.

2. The system of claim 1, wherein the step of calculating is performed using one of a group lasso criterion and a lasso criterion.

3. The system of claim 1, wherein each of said plurality of groups consists of at least one and no more than three markers.

4. The system of claim 1, wherein, for each of said plurality of groups, a number of markers in the respective group defines an upper limit for a number of stains per pixel of said plurality of pixels.

5. The system of claim 1, wherein each of said plurality of markers has an affinity to at least one tissue feature selected from the group consisting of a tumor cell cytokeratin, a regulatory T-cell nucleus, a universal nucleus, a B-cell membrane, a universal T-cell membrane, and a cytotoxic T-cell membrane.

6. A method of analyzing tissue obtained from a biological specimen, comprising:

storing, for each of a plurality of markers, color data indicative of a color of tissue marked by the respective marker;

storing co-location data defining a plurality of groups of said markers, each group consisting of markers having an affinity to a respective common tissue feature, wherein a tissue feature is a characteristic of a tissue that is indicative of a medical condition;

storing a plurality of pixels representative of a tissue image, each pixel comprising color information;

unmixing said tissue image using said color data, said co-location data and said pixels by calculating, for each of said pixels and for each of said groups, a linear combination of the colors of the markers of the respective group that yields a minimum difference between said color information of the respective pixel and said linear combination of colors;

determining, for each of said pixels, a group for which said minimum difference is smallest; and

outputting said tissue feature of said group as an analysis result.

7. The method of claim 6, further comprising:

staining a tissue sample with each of said plurality of markers; and

imaging said stained tissue sample to obtain said plurality of pixels representative of a tissue image.

8. The method of claim 6, wherein each of said plurality of groups consists of at least one and no more than three markers.

9. The method of claim 6, wherein, for each of said plurality of groups, a number of markers in the respective group defines an upper limit for a number of stains per pixel of said plurality of pixels.

10. The method of claim 6, wherein each of said plurality of markers has an affinity to at least one tissue feature selected from the group consisting of a tumor cell cytokeratin, a regulatory T-cell nucleus, a universal nucleus, a B-cell membrane, a universal T-cell membrane, and a cytotoxic T-cell membrane.