GROUP SPARSITY MODEL FOR IMAGE UNMIXING

• US 20160358335A1
• Filed: 08/22/2016
• Published: 12/08/2016
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: Ventana Medical Systems Incorporated (Roche Holding AG)
• Original Assignee: Ventana Medical Systems Incorporated (Roche Holding AG)
• Inventors: Chukka, Srinivas, Chen, Ting
• Granted Patent: US 10,540,762 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/698 Matching; Classification

Abstract

Systems and methods described herein relate, among other things, to unmixing more than three stains, while preserving the biological constraints of the biomarkers. Unlimited numbers of markers may be unmixed from a limited-channel image, such as an RGB image, without adding any mathematical complicity to the model. Known co-localization information of different biomarkers within the same tissue section enables defining fixed upper bounds for the number of stains at one pixel. A group sparsity model may be leveraged to explicitly model the fractions of stain contributions from the co-localized biomarkers into one group to yield a least squares solution within the group. A sparse solution may be obtained among the groups to ensure that only a small number of groups with a total number of stains being less than the upper bound are activated.

Claims

1. A method for spectral unmixing of an image obtained from a biological tissue sample being stained by multiple stains using a group Lasso criterion, the method comprising:

inputting image data obtained from the biological tissue sample;

reading reference data from a memory, the reference data being descriptive of the stain color of each one of the multiple stains;

reading colocation data from the memory, the colocation data being descriptive of groups of the stains, each group comprising stains that can be collocated in the biological tissue sample, and each group forming a group for the group Lasso criterion, at least one of the groups having a size of two or above;

calculating a solution of the group Lasso criterion for obtaining the unmixed image using the reference data as a reference matrix.

2. The method of claim 1, wherein a tuning parameter for a regularization term of the group Lasso criterion is determined by cross-validation.

3. The method of claim 1, wherein the biological tissue sample is stained by a first number of stains, each stain having a different stain color, and further comprising acquiring the image data via a second number of color channels, the second number being below the first number.

4. A tissue analysis system comprising:

a color data storage module to store, for each of a plurality of markers, color data indicative of a color of tissue marked by the respective marker;

a co-location data storage module to store co-location data defining a plurality of groups of said markers, each group consisting of markers having an affinity to a respective common tissue feature;

a tissue image data storage module to store a plurality of pixels representative of a tissue image, each pixel comprising color information; and

a tissue image analysis module to unmix said tissue image, wherein said tissue image analysis module is configured to read said color data from said color data storage module, said co-location data from said co-location data storage module and said pixels from said tissue image data storage module, and to calculate, for each of said pixels and for each of said groups, a linear combination of the colors of the markers of the respective group that yields a minimum difference between said color information of the respective pixel and said linear combination of colors, and wherein, for each of said pixels, said tissue image analysis module is to determine a group for which said minimum difference is smallest and outputs said tissue feature of said group as an analysis result.

5. The tissue analysis system of claim 4, wherein the step of calculating is performed using one of a group lasso criterion and a lasso criterion.

6. The tissue analysis system of claim 4, wherein each of said plurality of groups consists of at least one and no more than three markers.

7. The tissue analysis system of claim 4, wherein, for each of said plurality of groups, the number of markers in the respective group defines an upper limit for a number of stains per pixel of said plurality of pixels.

8. The tissue analysis system of claim 4, wherein each of said plurality of markers has an affinity to at least one tissue feature selected from the group consisting of a tumor cell cytokeratin, a regulatory T-cell nucleus, a universal nucleus, a B-cell membrane, a universal T-cell membrane, and a cytotoxic T-cell membrane.

9. A tissue analysis method, comprising:

storing, for each of a plurality of markers, color data indicative of a color of tissue marked by the respective marker;

storing co-location data defining a plurality of groups of said markers, each group consisting of markers having an affinity to a respective common tissue feature;

storing a plurality of pixels representative of a tissue image, each pixel comprising color information;

unmixing said tissue image using said color data, said co-location data and said pixels by calculating, for each of said pixels and for each of said groups, a linear combination of the colors of the markers of the respective group that yields a minimum difference between said color information of the respective pixel and said linear combination of colors;

determining, for each of said pixels, a group for which said minimum difference is smallest; and

outputting said tissue feature of said group as an analysis result.

10. The tissue analysis method of claim 9, further comprising:

staining a tissue sample with each of said plurality of markers; and

imaging said stained tissue sample to obtain said plurality of pixels representative of a tissue image.

11. The tissue analysis method of claim 9, wherein each of said plurality of groups consists of at least one and no more than three markers.

12. The tissue analysis method of claim 9, wherein, for each of said plurality of groups, the number of markers in the respective group defines an upper limit for a number of stains per pixel of said plurality of pixels.

13. The tissue analysis method of claim 9, wherein each of said plurality of markers has an affinity to at least one tissue feature selected from the group consisting of a tumor cell cytokeratin, a regulatory T-cell nucleus, a universal nucleus, a B-cell membrane, a universal T-cell membrane, and a cytotoxic T-cell membrane.

14. A non-transitory digital storage medium storing instructions executable by a processor to perform operations comprising:

generating a group sparsity model wherein a fraction of a stain contribution from colocation markers is assigned within a single group and a fraction of a stain contribution from non-colocation markers is assigned within separate groups; and

solving the group sparsity model using an unmixing algorithm to yield a least squares solution within each group.

15. The non-transitory digital storage medium of claim 14, wherein known co-location information of the colocation and non-colocation markers within the image enables defining fixed upper bounds for the number of stains at one pixel of the image.

16. The non-transitory digital storage medium of claim 14, wherein the group sparsity model is a group lasso model.

17. A system for unmixing an image, comprising:

a processor; and

a memory coupled to the processor, the memory to store digitally encoded instructions that, when executed by the processor, cause the processor to perform operations comprising;

generating a group sparsity framework using known co-location information of a plurality of biomarkers within an image of a tissue section;

wherein a fraction of each stain contribution is assigned to a different group based on the known co-location information; and

solving the group sparsity model using an unmixing algorithm to yield a least squares solution for each group.

18. The system of claim 17, wherein the known co-location information enables defining a fixed upper bound for the number of stains at a single pixel of the image.

19. The system of claim 17, wherein the operations further comprise ensuring sparsity among the groups but non-sparsity within each group.

20. A non-transitory storage medium storing instructions executable by a processor to cause the processor to perform operations comprising:

modeling an RGB image unmixing problem using a group sparsity framework, in which fractions of stain contributions from a plurality of colocation markers are modeled within a same group and fractions of stain contributions from a plurality of non-colocation markers are modeled in different groups;

providing co-localization information of the plurality of colocation markers to the modeled group sparsity framework;

solving the modeled framework using a group lasso to yield a least squares solution within each group, wherein the least squares solution corresponds to the unmixing of the colocation markers; and

yielding a sparse solution among the groups that corresponds to the unmixing of the non-colocation markers.