CLASSIFYING MEDICAL IMAGES USING DEEP CONVOLUTION NEURAL NETWORK (CNN) ARCHITECTURE

• US 20190073569A1
• Filed: 09/07/2017
• Published: 03/07/2019
• Est. Priority Date: 09/07/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ben-Ari, Rami, Kisilev, Pavel, Sulam, Jeremias
• Granted Patent: US 10,650,286 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/0091 for mammography
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: A61B 5/7282 Event detection, e.g. detecting unique waveforms indicative of a medical condition A61B5/4094, A61B5/4818, A61B5/349, A61B5/0823 take precedence
• CPC Class Codes: A61B 6/502 for diagnosis of breast, i.e. mammography
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Embodiments of the present systems and methods may provide the capability to classify medical images, such as mammograms, in an automated manner using existing annotation information. In embodiments, only the global, image level tag may be needed to classify a mammogram into certain types, without fine annotation of the findings in the image. In an embodiment, a computer-implemented method for classifying medical images may comprise receiving a plurality of image tiles, each image tile including a portion of a whole view, processed by a trained or a pre-trained model and outputting a one-dimensional feature vector for each tile to generate a three-dimensional feature volume and classifying the larger image by a trained model based on the generated three-dimensional feature volume to form a classification of the image.

Claims

1. A computer-implemented method for classifying medical images with a global image-level tag and without local annotations that reveal a position and a shape of findings within the image, comprising:

receiving a plurality of image tiles from a whole image, each image tile including a portion of the whole image, processed by a trained or a pre-trained model and outputting a one-dimensional feature vector for each tile to generate a three-dimensional feature volume; and

classifying the whole image by a trained model based on the generated three-dimensional feature volume to form a classification of the image.

2. The method of claim 1, wherein the whole image is one of a mammogram image or a breast Magnetic Resonance Imaging image.

3. The method of claim 2, wherein the classification is a binary classification that classifies the images into malignant or benign.

4. The method of claim 3, wherein the class populations are highly imbalanced.

5. The method of claim 3, wherein the three-dimensional feature volume is generated using a plurality of layers of convolutional network processing in a deep convolutional neural network, wherein the deep convolutional neural network uses a deep neural network loss function that directly maximizes an area under the curve (AUC) performance measure.

6. The method of claim 5, wherein the deep convolutional neural network combines a tile based approach with pre-trained encoding and trained convolutional neural network layers.

7. The method of claim 1, wherein sizes of the tiles are determined based on an expected size of findings in the images.

8. A system for classifying medical images with a global image-level tag and without local annotations that reveal a position and a shape of findings within the image, the system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

receiving a plurality of image tiles from a whole image, each image tile including a portion of the whole image, processed by a trained or a pre-trained model and outputting a one-dimensional feature vector for each tile to generate a three-dimensional feature volume; and

classifying the whole image by a trained model based on the generated three-dimensional feature volume to form a classification of the image.

9. The system of claim 8, wherein the whole image is one of a mammogram image or a breast Magnetic Resonance Imaging image.

10. The system of claim 9, wherein the classification is a binary classification that classifies the images into malignant or benign.

11. The system of claim 10, wherein the class populations are highly imbalanced.

12. The system of claim 10, wherein the three-dimensional feature volume is generated using a plurality of layers of convolutional network processing in a deep convolutional neural network, wherein the deep convolutional neural network uses a deep neural network loss function that directly maximizes an area under the curve (AUC) performance measure.

13. The system of claim 12, wherein the deep convolutional neural network combines a tile based approach with pre-trained encoding and trained convolutional neural network layers.

14. The system of claim 8, wherein sizes of the tiles are determined based on an expected size of findings in the images.

15. A computer program product for classifying medical images with a global image-level tag and without local annotations that reveal a position and a shape of findings within the image, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

receiving a plurality of image tiles from a whole image, each image tile including a portion of the whole image, processed by a trained or a pre-trained model and outputting a one-dimensional feature vector for each tile to generate a three-dimensional feature volume; and

classifying the whole image by a trained model based on the generated three-dimensional feature volume to form a classification of the image.

16. The computer program product of claim 15, wherein the whole image is one of a mammogram image or a breast Magnetic Resonance Imaging image.

17. The computer program product of claim 16, wherein the classification is a binary classification that classifies the images into malignant or benign.

18. The computer program product of claim 17, wherein the class populations are highly imbalanced.

19. The computer program product of claim 17, wherein the three-dimensional feature volume is generated using a plurality of layers of convolutional network processing in a deep convolutional neural network, wherein the deep convolutional neural network uses a deep neural network loss function that directly maximizes an area under the curve (AUC) performance measure.

20. The computer program product of claim 19, wherein the deep convolutional neural network combines a tile based approach with pre-trained encoding and trained convolutional neural network layers.