SYSTEMS AND METHODS FOR SEGMENTING DIGITAL IMAGES

• US 20120033862A1
• Filed: 08/06/2010
• Published: 02/09/2012
• Est. Priority Date: 08/06/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Wang, Su, Dai, Shengyang, Nakamura, Akira, Ohashi, Takeshi, Yokono, Jun
• Granted Patent: US 8,345,976 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

Methods and systems disclosed herein provide the capability to automatically process digital pathology images quickly and accurately. According to one embodiment, an digital pathology image segmentation task may be divided into at least two parts. An image segmentation task may be carried out utilizing both bottom-up analysis to capture local definition of features and top-down analysis to use global information to eliminate false positives. In some embodiments, an image segmentation task is carried out using a “pseudo-bootstrapping” iterative technique to produce superior segmentation results. In some embodiments, the superior segmentation results produced by the pseudo-bootstrapping method are used as input in a second segmentation task that uses a combination of bottom-up and top-down analysis.

Claims

1. A computer-implemented segmentation method for segmenting a digitized pathology image of tissue into at least two non-overlapping_regions for use in disease diagnosis, the method comprising:

generating a first initial data set comprising a segmentation of pixels in the digitized pathology image as belonging to a first region using a first method of processing;

generating a second initial data set comprising a current best segmentation of pixels in the digitized pathology image as belonging to the first region using a second method of processing;

iteratively determining a final first region data set based on the first and second initial data sets, wherein the final first region data set comprises a segmentation of pixels in the digitized pathology image as belonging to the first region; and

segmenting the digitized pathology image into a first region and a remainder region based on the final first region data set.

2. The method of claim 1, wherein the first method of processing comprises the steps of:

performing anisotropic diffusion on the digitized pathology image;

performing binary thresholding on results of the anisotropic diffusion step; and

performing connected-component analysis on results of the binary thresholding step.

3. The method of claim 1, wherein the second method of processing comprises the steps of:

generating a confidence map of a second region through feature extraction;

generating a first binary confidence map based on thresholding the confidence map;

applying the binary confidence map as an image mark to the digital pathology image to suppress the pixels of the second region to a constant value;

generating a second digitized pathology image with pixels of the second region set to a constant value; and

generating the second initial data set comprising a segmentation of pixels in the digitized, pathology image as belonging to the first region using a bottom-up processing method applied to the second digitized pathology image.

4. The method of claim 3, wherein iteratively determining the final first region data set based on the first and second initial data sets comprises the steps of:

generating a second binary confidence map based on thresholding the confidence map;

storing data of the first initial data set in a first iterative data set;

storing data of the second initial data set in a second iterative data set;

performing, until a threshold is met, an iterative loop comprising the steps of;

comparing the first iterative data set, the second iterative data set, and the second binary confidence map to produce a temporary image;

generating a bootstrap data set by processing the temporary image with bottom-up processing;

overwriting data of the first iterative data set with data of the second iterative data set; and

overwriting data of the second iterative data set with data of the bootstrap data set.

5. The method of claim 1, further comprising the steps of:

generating a top-down confidence map of the remainder region, wherein the top-down confidence map represents the likelihood that a pixel in the remainder does not represent an area of a second region;

determining a final third region data set from the confidence map and the final first region data set;

generating a final second region data set by subtracting the final first region data set and the final third region data set from a foreground of the digitized pathology image; and

segmenting the remainder region into second and third non-overlapping regions based on the final second and final third region data sets.

6. The method of claim 5, wherein determining a final third region data set from the confidence map and the final first region data set comprises the steps of:

determining a temporary fourth region data set and a temporary second region data set based on the top-down confidence map and the remainder of the tissue, wherein each temporary data set stores values representing whether a pixel of the digitized pathology image belongs to either the second region or fourth region, subtracting the temporary fourth region data set from the final first region data set to obtain a temporary third region data set; and

polishing the temporary third region data set to obtain the final third region data set.

7. The method of claim 6, wherein

the first region is a nucleus region;

the second region is a stroma region;

the third region is a goblet-lumen region; and

the fourth region is a gland region.

8. A computer-implemented information fusion method for segmenting a digitized pathology image of tissue into regions for use in disease diagnosis, the method comprising:

generating a confidence map of an unwanted region of the tissue by feature extraction;

obtaining a binary confidence map by thresholding the confidence map;

applying the binary confidence map as an image mark to the digital pathology image to suppress the unwanted region, wherein suppressing the unwanted region comprises changing pixel values of pixels in the unwanted region to a constant value;

generating a version of the digitized pathology image with pixels of the unwanted region set to a constant value; and

generating a data set comprising a segmentation of pixels in the digitized pathology image as belonging to the first region using a bottom-up processing method.

9. A system for segmenting a digital image into at least two segments comprising:

a memory having program instructions and data storage space;

a processor configured to use the program instructions to perform the steps of;

generating a first initial data set comprising a segmentation of pixels in the digitized pathology image as belonging to a first region using a first method of processing;

generating a second initial data set comprising a current best segmentation of pixels in the digitized pathology image as belonging to the first region using a second method of processing;

iteratively determining a final first region data set based on the first and second initial data sets, wherein the final first region data set comprises a segmentation of pixels in the digitized pathology image as belonging to the first region; and

segmenting the digitized pathology image into a first region and a remainder region based on the final first region data set.

10. The system as recited in claim 9, wherein the processor configured for performing the first method of processing is further configured for:

performing anisotropic diffusion on the digitized pathology image;

performing binary thresholding on results of the anisotropic diffusion step; and

performing connected-component analysis on results of the binary thresholding step.

11. The system as recited in claim 9, wherein the processor configured for performing the second method of processing is further configured for:

generating a confidence map of a second region through feature extraction;

generating a first binary confidence map based on thresholding the confidence map;

applying the binary confidence map as an image mark to the digital pathology image to suppress the pixels of the second region to a constant value;

generating a second digitized pathology image with pixels of the second region set to a constant value; and

generating the second initial data set comprising a segmentation of pixels in the digitized pathology image as belonging to the first region using a bottom-up processing method applied to the second digitized pathology image.

12. The system as recited in claim 11, wherein the processor configured for iteratively determining the final first region data set based on the first and second initial data sets is further configured for:

generating a second binary confidence map based on thresholding the confidence map;

storing data of the first initial data set in a first iterative data set;

storing data of the second initial data set in a second iterative data set;

performing, until a threshold is met, an iterative loop comprising the steps of;

comparing the first iterative data set, the second iterative data set, and the second binary confidence map to produce a temporary image;

generating a bootstrap data set by processing the temporary image with bottom-up processing;

overwriting data of the first iterative data set with data of the second iterative data set; and

overwriting data of the second iterative data set with data of the bootstrap data set.

13. The system as recited in claim 9, wherein the processor is further configured for:

generating a top-down confidence map of the remainder region, wherein the top-down confidence map represents the likelihood that a pixel in the remainder does not represent an area of a second region;

determining a final third region data set from the confidence map and the final first region data set;

generating a final second region data set by subtracting the final first region data set and the final third region data set from a foreground of the digitized pathology image; and

segmenting the remainder region into second and third non-overlapping regions based on the final second and final third region data sets.

14. The system as recited in claim 13, wherein the processor configured for determining a final third region data set from the confidence map and the final first region data set, is further configured for:

determining a temporary fourth region data set and a temporary second region data set based on the top-down confidence map and the remainder of the tissue, wherein each temporary data set stores values representing whether a pixel of the digitized pathology image belongs to either the second region or fourth region,

subtracting the temporary fourth region data set from the final first region data set to obtain a temporary third region data set; and

polishing the temporary third region data set to obtain the final third region data set.

15. The system as recited in claim 14, wherein

the first region is a nucleus region;

the second region is a stroma region;

the third region is a goblet-lumen region; and

the fourth region is a gland region.