Systems and methods for automatic detection of architectural distortion in two dimensional mammographic images

• US 10,037,601 B1
• Filed: 02/02/2017
• Issued: 07/31/2018
• Est. Priority Date: 02/02/2017
• Status: Active Grant

• Current Assignee: Merative US LP
• Original Assignee: International Business Machines Corporation
• Inventors: Ben-Ari, Rami
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/422,483
• Publication Number:
  US 20180218495A1
• Time in Patent Office: 544 Days
• Field of Search: 382128, 382171, 382130, 378 37, 705 3, 707999006
• CPC Class Codes: A61B 6/502 for diagnosis of breast, i.e. mammography
• CPC Class Codes: A61B 6/5258 involving detection or reduction of artifacts or noise
• CPC Class Codes: G06F 18/10 Pre-processing; Data cleansing
• CPC Class Codes: G06F 18/2113 by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 5/75 Unsharp masking
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 : using an image reference approach
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06T 7/136 : involving thresholding
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/267 : by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/454 : Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 : using neural networks
• CPC Class Codes: G06V 2201/03 : Recognition of patterns in medical or anatomical images

Abstract

There is provided a method, comprising: segmenting fibroglandular tissue of a 2D mammographic image of a breast, extracting regions within the segmented fibroglandular tissue and within a boundary portion between the segmented fibroglandular tissue and non-fibroglandular tissue, computing representations for each RoI by a pre-trained deep neural network, training a classifier on the representations to compute a probability score of architectural distortion for each RoI, clustering RoIs defined as positive for architectural distortion using a mean-shift method and providing an indication of the probability of the presence of architectural distortion around a cluster based on the probability distribution of cluster RoI members, removing small clusters having fewer RoI members than a small number threshold, classifying the image as positive for the indication of architectural distortion when at least one cluster remains, or classifying the image as negative for the indication of architectural distortion when no cluster remains.

Claims

1. A method for using a trained statistical classifier for detecting an indication of architectural distortion in a mammographic image, comprising:

receiving a two dimensional (2D) mammographic image of a breast;

segmenting fibroglandular tissue of the breast to create a segmented fibroglandular tissue region;

extracting a plurality of regions within the segmented fibroglandular tissue region and within a boundary portion between the segmented fibroglandular tissue and non-fibroglandular tissue;

computing representations for each region of interest (RoI) by a pre-trained deep neural network;

training a classifier on the computed representations to compute a respective probability score of architectural distortion associated with each RoI;

defining each RoI having the probability score above a threshold as positive for architectural distortion;

clustering the RoIs defined as positive using a mean-shift method and providing an indication of the probability of the presence of architectural distortion around a cluster based on a probability distribution of cluster RoI members;

removing small clusters created by the clustering of the RoI according to a small number threshold, wherein clusters having fewer RoI members than the small number threshold are removed;

classifying the image as positive for the indication of architectural distortion when at least one cluster remains after the removing, or classifying the image as negative for the indication of architectural distortion when no cluster remains after the removing; and

outputting a classification of the image.

2. The method of claim 1, further comprising when the image is classified as positive, applying a regression operation to the RoIs to obtain at least one localization and prediction for the RoI size indicative of the region within the 2D mammographic image where the architectural distortion is identified.

3. The method of claim 1, further comprising, when the image is classified as positive, applying a secondary mean-shift clustering method to the RoIs having probability scores above a second threshold, to select at least one localization RoI indicative of localization of the indication of the architectural distortion within the 2D mammographic image.

4. The method of claim 3, further comprising, marking on the 2D mammographic image, the location of the indication of the architectural distortion corresponding to the location of the at least one localization RoI.

5. The method of claim 3, further comprising computing a union of a plurality of the localization RoIs, the union indicative of the indication of the architectural distortion.

6. The method of claim 1, wherein the fibroglandular tissue is segmented by applying an unsharp mask filter to the mammographic image to obtain a first processed image, normalizing the values of the pixels of the first processed image, and applying threshold to the normalized values to obtain the segmented fibroglandular tissue.

7. The method of claim 1, wherein the RoIs are extracted by sparse and random sampling of the segmented fibroglandular tissue.

8. The method of claim 1, wherein negative RoIs are defined entirely within the segmented fibroglandular tissue.

9. The method of claim 1, wherein the mean shift method used for clustering the RoIs defined as positive is a non-parametric clustering technique that does not require prior knowledge of the number of clusters.

10. The method of claim 1, wherein the neural network is implemented as a deep neural network.

11. The method of claim 1, wherein the classifier is trained using deep neural network transfer learning that is based on using intermediate outputs of a pretrained neural network obtained from at least one lower layer.

12. The method of claim 1, wherein the classifier is implemented as a cascade classifier implemented as a linear support vector machine (SVM).

13. The method of claim 1, further comprising computing a symmetric overlap ratio used to validate the localization of architectural distortion, the symmetric overlap ratio denoted as:

14. A method for training a statistical classifier for detecting an indication of architectural distortion in a mammographic image of a breast, comprising:

receiving a set of training 2D mammographic member images including a sub-set of 2D mammographic images labeled as positive for architectural distortion, wherein the 2D mammographic images comprise screening mammographic images, wherein a size of the sub-set being inadequate for training a standard R-CNN to achieve statistically significant classification;

segmenting fibroglandular tissue of the breast of each member image to create a segmented fibroglandular tissue region;

extracting positive RoIs from regions around an identified architectural distortion of the sub-set of 2D mammographic images labeled as positive for architectural distortion;

extracting negative RoIs from random regions in the fibroglandular tissues of normal mammographic images that are not labeled as positive for architectural distortion;

computing representations for each RoI using a pre-trained neural network;

training a binary object cascade classifier, using the computed representations, to compute a respective probability score indicative of architectural distortion associated with each RoI; and

providing the trained binary object cascade classifier for classifying a new 2D mammographic image as positive for the indication of architectural distortion.

15. The method of claim 14, further comprising apply image augmentation to the positive RoIs.

16. The method of claim 14, wherein the size of the subset is less than 100 2D mammographic images labeled as positive for architectural distortion.

17. The method of claim 14, further comprising training a regressor to localize the architectural distortion, based on the RoIs classified as including architectural distortion by the classifier based on a high probability score above a threshold, wherein the RoIs classified as including architectural distortion overlap an annotated region defined as including the architectural distortion.

18. The method of claim 14, wherein the binary object cascade classifier is trained by returning samples incorrectly rejected at a first cascade level as input into a following cascade level.

19. The method of claim 14, wherein the training images labeled as positive for architectural distortion are annotated with a simple marking such as bounding box, defining a boundary of an annotated region containing the architectural distortion.

20. A system for using a trained statistical classifier for detecting an indication of architectural distortion in a mammographic image, comprising:

a program store storing code; and

a processor coupled to the program store for implementing the stored code, the code comprising;

code to receive a two dimensional (2D) mammographic image of a breast;

code to segment fibroglandular tissue of the breast to create a segmented fibroglandular tissue region, extract a plurality of regions within the segmented fibroglandular tissue region and within a boundary portion between the segmented fibroglandular tissue and non-fibroglandular tissue, compute representations for each RoI by a neural network trained on a plurality of sample 2D mammographic images using automatically identified and extracted features, apply a classifier to the computed representations to compute a respective probability score of architectural distortion associated with each RoI, define each RoI having the probability score above a threshold as positive for architectural distortion, cluster the RoIs defined as positive using a mean-shift method to provide an indication of the probability of the presence of architectural distortion for each respective RoI, remove small clusters created by the clustering of the RoI according to a small number threshold, wherein clusters having fewer RoI members than the small number threshold are removed, classify the image as positive for the indication of architectural distortion when at least one cluster remains after the removing, or classify the image as negative for the indication of architectural distortion when no cluster remains after the removing; and

output a classification of the image.