Identifying and excluding blurred areas of images of stained tissue to improve cancer scoring

• US 10,565,479 B1
• Filed: 10/04/2019
• Issued: 02/18/2020
• Est. Priority Date: 12/27/2016
• Status: Active Grant

• Current Assignee: Definiens GmbH
• Original Assignee: Definiens GmbH
• Inventors: Lesniak, Jan Martin
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US16/593,968
• Publication Number:
  US 20200034651A1
• Time in Patent Office: 137 Days
• Field of Search: None
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

A method for identifying blurred areas in digital images of stained tissue involves artificially blurring a learning tile and then training a pixel classifier to correctly classify each pixel as belonging either to the learning tile or to a blurred copy. A learning tile is first selected from a digital image of stained tissue. The learning tile is copied and blurred by applying a filter to each pixel. The pixel classifier is trained to correctly classify each pixel as belonging either to the learning tile or to the blurred, copied learning tile. The pixel classifier then classifies each pixel of the entire digital image as most likely resembling either the learning tile or the blurred learning tile. The digital image is segmented into blurred and unblurred areas based on the pixel classification. The blurred areas and the unblurred areas of the digital image are identified on a graphical user interface.

Claims

1. A method comprising:

selecting a learning region of a digital image of a slice of tissue from a cancer patient that has been stained using a biomarker, wherein the digital image comprises pixels, wherein each of the pixels has a color defined by pixel values, wherein a portion of the pixels exhibits the color stained using the biomarker, and wherein the learning region includes a first sub region and a second sub region;

distorting the second sub region of the learning region by applying a filter to the pixel values of each pixel of the second sub region so as artificially to blur the second sub region;

generating a pixelwise descriptor by analyzing and comparing the pixel values of each pixel of the learning region with the pixel values of neighboring pixels at predetermined offsets from each analyzed pixel, wherein the pixelwise descriptor is trained to indicate based on the comparing with neighboring pixels that each pixel of the learning region most likely belongs either to an unblurred class of pixels such as those in the first sub region or to a blurred class of pixels such as those in the second sub region;

characterizing each pixel of the digital image as most likely belonging either to the unblurred class of pixels or to the blurred class of pixels using the pixelwise descriptor by classifying each characterized pixel based on the pixel values of neighboring pixels at predetermined offsets from each characterized pixel; and

identifying blurred areas of the digital image based on the classifying of pixels as belonging to the blurred class of pixels;

generating image objects by segmenting the digital image except in the identified blurred areas;

determining a score using the image objects, wherein the score is indicative of a level of cancer malignancy of the slice of tissue from the cancer patient.

2. The method of claim 1, further comprising:

dividing the digital image into tiles, wherein the learning region is one of the tiles, and wherein for each pixel the color stained using the biomarker and defined by the pixel values has a magnitude derived from the pixel values; and

identifying the learning region as the tile whose pixel values represent a mean magnitude of the color stained using the biomarker.

3. The method of claim 1, further comprising:

performing object-oriented image analysis on the digital image except for in the identified blurred areas.

4. The method of claim 1, wherein each of the pixels has a color that is a shade of gray.

5. The method of claim 1, wherein the pixelwise descriptor is a set of random forest pixelwise descriptors.

6. The method of claim 1, further comprising:

determining a histopathological score based on tissue stained using the biomarker in areas of the digital image that are outside the blurred areas.

7. The method of claim 1, wherein the biomarker is used to immunohistochemically stain for the Ki-67 protein.

8. The method of claim 1, wherein the pixelwise descriptor is a binary classifier that is trained using supervised learning.

9. A method comprising:

selecting a region of a digital image of cancer tissue that has been stained using a biomarker, wherein the digital image comprises pixels, wherein each of the pixels has a color defined by pixel values, wherein a portion of the pixels exhibits the color stained using the biomarker, and wherein the region includes a first sub region and a second sub region;

distorting the second sub region by modifying the pixel values of each pixel of the second sub region so as artificially to blur the second sub region;

generating a pixelwise descriptor by comparing the pixel values of each pixel of the region with the pixel values of neighboring pixels at predetermined offsets from each analyzed pixel, wherein the pixelwise descriptor is trained to indicate based on the comparing that each pixel of the region most likely belongs either to an unblurred class of pixels such as those in the first sub region or to a blurred class of pixels such as those in the second sub region;

characterizing each pixel of the digital image as most likely belonging either to the unblurred class of pixels or to the blurred class of pixels using the pixelwise descriptor by classifying each characterized pixel based on the pixel values of neighboring pixels at predetermined offsets from each characterized pixel; and

identifying blurred areas of the digital image based on the classifying of pixels as belonging to the blurred class of pixels;

generating image objects by segmenting the digital image except in the identified blurred areas; and

determining a score using the image objects, wherein the score is indicative of a level of cancer malignancy of the cancer tissue.

10. The method of claim 9, further comprising:

dividing the digital image into tiles, wherein the region is one of the tiles, and wherein for each pixel the color stained using the biomarker and defined by the pixel values has a magnitude derived from the pixel values; and

identifying the region as the tile whose pixel values represent a mean magnitude of the color stained using the biomarker.

11. The method of claim 9, further comprising:

performing object-oriented image analysis on the digital image except for in the identified blurred areas.

12. The method of claim 9, wherein each of the pixels has a color that is a shade of gray.

13. The method of claim 9, wherein the pixelwise descriptor is a set of random forest pixelwise descriptors.

14. The method of claim 9, further comprising:

determining a histopathological score based on cancer tissue stained using the biomarker in areas of the digital image that are outside the identified blurred areas.

15. The method of claim 9, wherein the biomarker is used to immunohistochemically stain for the Ki-67 protein.

16. The method of claim 9, wherein the pixelwise descriptor is a binary classifier that is trained using supervised learning.