Identifying and Excluding Blurred Areas of Images of Stained Tissue To Improve Cancer Scoring

• US 20180182099A1
• Filed: 12/27/2016
• Published: 06/28/2018
• Est. Priority Date: 12/27/2016
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Lesniak, Jan Martin
• Granted Patent: US 10,438,096 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

A method for identifying blurred areas in digital images of stained tissue involves artificially blurring a learning tile and then training a pixel classifier to correctly classify each pixel as belonging either to the learning tile or to a blurred copy. A learning tile is first selected from a digital image of stained tissue. The learning tile is copied and blurred by applying a filter to each pixel. The pixel classifier is trained to correctly classify each pixel as belonging either to the learning tile or to the blurred, copied learning tile. The pixel classifier then classifies each pixel of the entire digital image as most likely resembling either the learning tile or the blurred learning tile. The digital image is segmented into blurred and unblurred areas based on the pixel classification. The blurred areas and the unblurred areas of the digital image are identified on a graphical user interface.

Claims

1. A method comprising:

selecting a learning region of a digital image of a slice of tissue from a cancer patient that has been stained using a biomarker, wherein the digital image comprises pixels, wherein each of the pixels has a color defined by pixel values, wherein a portion of the pixels exhibits the color stained using the biomarker;

duplicating the learning region to create a copied learning region;

distorting the copied learning region by applying a filter to the pixel values of each pixel of the copied learning region so as artificially to blur the copied learning region;

training a pixel classifier by analyzing the pixel values of each pixel of the learning region and the pixel values of a corresponding pixel in the copied learning region, wherein the pixel classifier is trained to correctly classify each pixel as belonging either to the learning region or to the copied learning region;

classifying each pixel of the digital image as most likely resembling either the learning region or the copied learning region using the pixel classifier;

segmenting the digital image into blurred areas and unblurred areas based on the classifying of each pixel as belonging either to the learning region or to the copied learning region; and

identifying the blurred areas and the unblurred areas of the digital image on a graphical user interface.

2. The method of claim 1, wherein the pixel classifier is taken from the group consisting of:

a random forest classifier, a convolutional neuronal network, a decision tree classifier, a support vector machine classifier and a Bayes classifier.

3. The method of claim 1, wherein the pixel values of the digital image include red, green and blue pixel values, and wherein the selecting of the learning region uses transformation pixel values obtained using a color transformation of the red, green and blue pixel values of the digital image.

4. The method of claim 1, wherein the analyzing the pixel values involves comparing the pixel values of each analyzed pixel with the pixel values of neighboring pixels at predetermined offsets from each analyzed pixel, and wherein based on the comparing the pixel classifier is trained to indicate that each pixel of the learning region and the copied learning region most likely belongs either to a blurred class of pixels such as those in the copied learning region or to an unblurred class of pixels such as those in the learning region.

5. The method of claim 1, further comprising:

segmenting the unblurred areas of the digital image to generate image objects;

determining a score using the image objects, wherein the score is indicative of a level of cancer malignancy of the slice of tissue from the cancer patient.

6. The method of claim 1, further comprising:

determining a histopathological score based on tissue stained using the biomarker in the unblurred areas of the digital image.

7. The method of claim 1, wherein the biomarker is used to immunohistochemically stain for the Ki-67 protein.

8. The method of claim 1, further comprising:

dividing the digital image into tiles, wherein the learning region is one of the tiles, and wherein for each pixel the color stained using the biomarker has a magnitude derived from the pixel values of the pixel; and

identifying the learning region as the tile whose pixel values represent a mean magnitude of the color stained using the biomarker.

9. The method of claim 1, wherein the pixel classifier is a binary classifier that is trained using supervised learning.

10. A method comprising:

selecting a learning region of a digital image of tissue that has been stained using a biomarker;

duplicating the learning region to create a copied learning region;

blurring the copied learning region by applying a filter to each pixel of the copied learning region;

generating a pixelwise descriptor by comparing learning pixels of the learning region and the copied learning region to neighboring pixels at predetermined offsets from each of the learning pixels, wherein based on the comparing the pixelwise descriptor is trained to indicate that each of the learning pixels most likely belongs either to an unblurred class of pixels such as those in the learning region or to a blurred class of pixels such as those in the copied learning region;

classifying each pixel of the digital image as belonging either to the unblurred class of pixels or to the blurred class of pixels using the pixelwise descriptor;

segmenting the digital image into blurred areas and unblurred areas based on the classifying; and

identifying the blurred areas and the unblurred areas of the digital image on a graphical user interface.

11. The method of claim 10, further comprising:

determining a histopathological score based on the tissue stained using the biomarker in the unblurred areas of the digital image.

12. The method of claim 10, wherein the biomarker is used to stain the tissue with a staining color, wherein a portion of the pixels of the digital image exhibits the staining color, further comprising:

determining a score based on image objects identified in the unblurred areas of the digital image using the pixels that exhibit the staining color, wherein the score is indicative of a level of cancer malignancy of the tissue.

13. The method of claim 10, further comprising:

segmenting the unblurred areas of the digital image into image objects; and

determining a score using the image objects, wherein the score is indicative of a level of cancer malignancy of the tissue.

14. The method of claim 10, wherein each pixel of the digital image has a pixel value, and wherein the filter applied to each pixel of the copied learning region is a Gaussian filter that modifies the value of each pixel based on the values of neighboring pixels.

15. The method of claim 14, wherein the Gaussian filter is applied at a radius of ten microns.

16. The method of claim 10, wherein the pixelwise descriptor is a binary classifier that is trained using supervised learning.

17. The method of claim 10, wherein each pixel of the digital image has a color defined by pixel values, and wherein the pixelwise descriptor compares the pixel values of each classified pixel of the digital image with the pixel values of neighboring pixels at the predetermined offsets from each classified pixel.

18. A method comprising:

selecting a learning region of a digital image of a slice of tissue from a cancer patient that has been stained using a biomarker, wherein the digital image comprises pixels, wherein each of the pixels has a color defined by pixel values, wherein a portion of the pixels exhibits the color stained using the biomarker, and wherein the learning region includes a first subregion and a second subregion;

distorting the second subregion of the learning region by applying a filter to the pixel values of each pixel of the second subregion so as artificially to blur the second subregion;

generating a pixelwise descriptor by analyzing and comparing the pixel values of each pixel of the learning region with the pixel values of neighboring pixels at predetermined offsets from each analyzed pixel, wherein the pixelwise descriptor is trained to indicate based on the comparing with neighboring pixels that each pixel of the learning region most likely belongs either to an unblurred class of pixels such as those in the first subregion or to a blurred class of pixels such as those in the second subregion;

characterizing each pixel of the digital image as most likely belonging either to the unblurred class of pixels or to the blurred class of pixels using the pixelwise descriptor by classifying each characterized pixel based on the pixel values of neighboring pixels at predetermined offsets from each characterized pixel; and

identifying blurred areas of the digital image based on the classifying of pixels as belonging to the blurred class of pixels.

19. The method of claim 18, further comprising:

dividing the digital image into tiles, wherein the learning region is one of the tiles, and wherein for each pixel the color stained using the biomarker and defined by the pixel values has a magnitude derived from the pixel values; and

identifying the learning region as the tile whose pixel values represent a mean magnitude of the color stained using the biomarker.

20. The method of claim 18, further comprising:

performing object-oriented image analysis on the digital image except for in the identified blurred areas.

21. The method of claim 18, further comprising:

generating image objects by segmenting the digital image except in the identified blurred areas;

determining a score using the image objects, wherein the score is indicative of a level of cancer malignancy of the slice of tissue from the cancer patient.

22. The method of claim 18, wherein each of the pixels has a color that is a shade of gray.

23. A method comprising:

selecting a learning region of a digital image of a slice of tissue from a cancer patient that has been stained using a biomarker, wherein the digital image comprises pixels, wherein each of the pixels has a color defined by pixel values, wherein a portion of the pixels exhibits the color stained using the biomarker;

duplicating the learning region to create a copied learning region;

distorting the copied learning region by applying a filter to the pixel values of each pixel of the copied learning region so as artificially to blur the copied learning region;

training a pixel classifier by analyzing the pixel values of each pixel of the learning region and the pixel values of a corresponding pixel in the copied learning region, wherein the pixel classifier is trained to correctly classify each pixel as belonging either to the learning region or to the copied learning region;

determining a metric indicative of the pixels of the digital image that most likely resemble the learning region; and

displaying a warning indicator on a graphical user interface when the metric is less than a predetermined threshold.

24. The method of claim 23, wherein the metric indicates a percentage of pixels of the digital image that most likely resemble the learning region as a proportion of a total number of pixels that resemble the learning region plus the pixels that resemble the copied learning region.