Deep convolutional neural network with self-transfer learning

• US 10,460,440 B2
• Filed: 10/24/2017
• Issued: 10/29/2019
• Est. Priority Date: 10/24/2017
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Zhang, Min, Avinash, Gopal Biligeri
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/792,698
• Publication Number:
  US 20190122360A1
• Time in Patent Office: 735 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

Systems and techniques for facilitating a deep convolutional neural network with self-transfer learning are presented. In one example, a system includes a machine learning component, a medical imaging diagnosis component and a visualization component. The machine learning component generates learned medical imaging output regarding an anatomical region based on a convolutional neural network that receives medical imaging data. The machine learning component also performs a plurality of sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The medical imaging diagnosis component determines a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network. The visualization component generates a multi-dimensional visualization associated with the classification and the localization for the portion of the anatomical region.

Claims

1. A convolutional neural network system, comprising:

a memory that stores computer executable components;

a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise;

a machine learning component that generates learned medical imaging output regarding an anatomical region based on a convolutional neural network that receives medical imaging data, wherein the machine learning component performs a first convolutional layer process associated with sequential downsampling of the medical imaging data followed by a second convolutional layer process associated with sequential upsampling of the medical imaging data, and wherein a first convolutional layer of the first convolutional layer process corresponds to a last convolutional layer of the second convolutional layer process;

a medical imaging diagnosis component that determines a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network; and

a visualization component that generates a multi-dimensional visualization associated with the classification and the localization for the portion of the anatomical region.

2. The convolutional neural network system of claim 1, wherein the machine learning component performs the first convolutional layer process based on a first convolutional layer filter that comprises a first size and a second convolutional layer filter that comprises a second size that is different than the first size, and wherein the machine learning process performs the second convolution layer process based on the first convolutional layer filter that comprises the first size and the second convolutional layer filter that comprises the second size.

3. The convolutional neural network system of claim 1, wherein the machine learning component performs a local pooling process for an activation map associated with a convolutional layer of the convolutional neural network prior to performing a global pooling process associated with the convolutional neural network.

4. The convolutional neural network system of claim 1, wherein the machine learning component performs the sequential upsampling of the medical imaging data in a reverse sampling sequence with respect to the sequential downsampling of the medical imaging data.

5. The convolutional neural network system of claim 1, wherein the machine learning component generates the learned medical imaging output based on a class activation mapping process that applies a set of weights to a set of heat maps associated with the medical imaging data.

6. The convolutional neural network system of claim 1, wherein the machine learning component merges a set of classifier layers associated with the convolutional neural network and a set of activation maps associated with the convolutional neural network to generate the learned medical imaging output.

7. The convolutional neural network system of claim 1, wherein the machine learning component processes the medical imaging data based on one or more regularization techniques to classify one or more portions of the medical imaging data.

8. The convolutional neural network system of claim 1, wherein the machine learning component employs a first portion of the medical imaging data for training associated with the convolutional neural network, a second portion of the medical imaging data for validation associated with the convolutional neural network, and a third portion of the medical imaging data for testing associated with the convolutional neural network.

9. The convolutional neural network system of claim 1, wherein the machine learning component randomly selects a set of medical images from a training set associated with the medical imaging data for data augmentation associated with the medical imaging data.

10. The convolutional neural network system of claim 9, wherein the machine learning component modifies an orientation of the set of medical images for the data augmentation associated with the medical imaging data.

11. The convolutional neural network system of claim 9, wherein the machine learning component modifies the set of medical images through at least one affine transformation for the data augmentation associated with the medical imaging data.

12. A method, comprising:

receiving, by a system comprising a processor, medical imaging data for a patient body;

performing, by the system, iterative sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of a convolutional neural network to generate learned medical imaging output regarding the patient body, wherein a first convolutional layer for the downsampling corresponds to a last convolutional layer for the upsampling;

classifying, by the system, a disease for a portion of the patient body based on the learned medical imaging output associated with the convolutional neural network; and

generating, by the system, a multi-dimensional visualization associated with the classifying of the disease for the portion of the patient body.

13. The method of claim 12, wherein the performing the iterative sequential downsampling and upsampling of the medical imaging data comprises:

analyzing the medical imaging data based on a first filter that comprises a first size;

analyzing the medical imaging data based on a second filter that comprises a second size that is different than the first size; and

analyzing the medical imaging data based on a third filter that comprises the first size associated with the first filter.

14. The method of claim 12, further comprising:

performing, by the system, a local pooling process for an activation map associated with a convolutional layer of the convolutional neural network prior to performing a global pooling process associated with the convolutional neural network.

15. The method of claim 12, wherein the performing the iterative sequential downsampling and upsampling of the medical imaging data comprises generating the learned medical imaging output based on a first convolutional layer process associated with downsampling of the medical imaging data and a second convolutional layer process associated with upsampling of the medical imaging data.

16. The method of claim 12, further comprising:

generating, by the system, the learned medical imaging output based on a class activation mapping process that applies a set of weights to a set of heat maps associated with the medical imaging data.

17. The method of claim 12, further comprising:

merging, by the system, a set of classifier layers associated with the convolutional neural network and a set of activation maps associated with the convolutional neural network to generate the learned medical imaging output.

18. A method, comprising:

receiving, by a system comprising a processor, medical imaging data that comprises a set of medical images;

training, by the system, a convolutional neural network by performing a first convolutional layer process associated with downsampling of the medical imaging data followed by a second convolutional layer process associated with upsampling of the medical imaging data, wherein an initial convolutional layer of the first convolutional layer process corresponds to a last convolutional layer of the second convolutional layer process; and

generating, by the system, a set of filter values for the convolutional neural network based on the first convolutional layer process associated with the downsampling and the second convolutional layer process associated with the upsampling.

19. The method of claim 18, wherein the training comprises:

analyzing the medical imaging data based on a first convolutional layer filter that comprises a first size;

analyzing the medical imaging database on a second convolutional layer filter that comprises a second size that is different than the first size; and

analyzing the medical imaging database on a third convolutional layer filter that comprises the first size associated with the first convolutional layer filter.

20. The method claim 18, wherein the generating the set of filter values comprises generating a set of weights for the convolutional neural network based on the first convolutional layer process associated with the downsampling and the second convolutional layer process associated with the upsampling.