Deep convolutional neural network with self-transfer learning

• US 11,074,687 B2
• Filed: 09/17/2019
• Issued: 07/27/2021
• Est. Priority Date: 10/24/2017
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Zhang, Min, Avinash, Gopal Biligeri
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US16/573,222
• Publication Number:
  US 20200013165A1
• Time in Patent Office: 679 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30008 Bone
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

Systems and techniques for facilitating a deep convolutional neural network with self-transfer learning are presented. In one example, a system includes a machine learning component, a medical imaging diagnosis component and a visualization component. The machine learning component generates learned medical imaging output regarding an anatomical region based on a convolutional neural network that receives medical imaging data. The machine learning component also performs a plurality of sequential downsampling and upsampling of the medical imaging data associated with convolutional layers of the convolutional neural network. The medical imaging diagnosis component determines a classification and an associated localization for a portion of the anatomical region based on the learned medical imaging output associated with the convolutional neural network. The visualization component generates a multi-dimensional visualization associated with the classification and the localization for the portion of the anatomical region.

Claims

1. A convolutional neural network system, comprising:

a memory that stores computer executable components;

a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise;

a machine learning component that performs a first convolutional layer process associated with sequential downsampling of medical imaging data followed by a second convolutional layer process associated with sequential upsampling of the medical imaging data to facilitate generation of learned medical imaging output regarding an anatomical region, wherein a first convolutional layer of the first convolutional layer process corresponds to a last convolutional layer of the second convolutional layer process; and

a medical imaging diagnosis component that predicts a medical condition associated with the anatomical region based on the learned medical imaging output.

2. The convolutional neural network system of claim 1, wherein the machine learning component employs a convolutional neural network associated with the first convolutional layer process and the second convolutional layer process to generate the learned medical imaging output.

3. The convolutional neural network system of claim 1, wherein the machine learning component performs the first convolutional layer process based on a first convolutional layer filter that comprises a first size and a second convolutional layer filter that comprises a second size that is different than the first size, and wherein the machine learning component performs the second convolutional layer process based on the first convolutional layer filter that comprises the first size and the second convolutional layer filter that comprises the second size.

4. The convolutional neural network system of claim 1, wherein the machine learning component performs the sequential upsampling of the medical imaging data in a reverse sampling sequence with respect to the sequential downsampling of the medical imaging data.

5. The convolutional neural network system of claim 1, wherein the machine learning component generates the learned medical imaging output based on a class activation mapping process that applies a set of weights to a set of heat maps associated with the medical imaging data.

6. The convolutional neural network system of claim 1, wherein the medical imaging diagnosis component determines a classification for the medical condition associated with the anatomical region based on the learned medical imaging output.

7. The convolutional neural network system of claim 1, wherein the medical imaging diagnosis component determines a localization for at least a portion of the anatomical region associated with the medical condition based on the learned medical imaging output.

8. The convolutional neural network system of claim 1, wherein the computer executable components comprise:

a visualization component that generates a multi-dimensional visualization related to the medical condition associated with the anatomical region.

9. A method, comprising:

performing, by a system comprising a processor, iterative sequential downsampling and upsampling of medical imaging data associated with convolutional layers of a convolutional neural network to generate learned medical imaging output regarding a patient body, wherein a first convolutional layer for the downsampling corresponds to a last convolutional layer for the upsampling; and

predicting, by the system, a medical condition associated with a portion of the patient body based on the learned medical imaging output.

10. The method of claim 9, wherein the performing the iterative sequential downsampling and upsampling of the medical imaging data comprises:

analyzing the medical imaging data based on a first filter that comprises a first size;

analyzing the medical imaging data based on a second filter that comprises a second size that is different than the first size; and

analyzing the medical imaging data based on a third filter that comprises the first size associated with the first filter.

11. The method of claim 9, wherein the performing the iterative sequential downsampling and upsampling of the medical imaging data comprises generating the learned medical imaging output based on a first convolutional layer process associated with the downsampling of the medical imaging data and a second convolutional layer process associated with the upsampling of the medical imaging data.

12. The method of claim 9, wherein the performing the iterative sequential downsampling and upsampling of the medical imaging data comprises performing the upsampling of the medical imaging data in a reverse sampling sequence with respect to the downsampling of the medical imaging data.

13. The method of claim 9, wherein the predicting the medical condition associated with the portion of the patient body comprises determining a classification for the medical condition associated with the portion of the patient body based on the learned medical imaging output.

14. The method of claim 9, wherein the predicting the medical condition associated with the portion of the patient body comprises determining a localization for the portion of the patient body associated with the medical condition based on the learned medical imaging output.

15. The method of claim 12, further comprising:

generating, by the system, a multi-dimensional visualization related to the medical condition associated with the portion of the patient body.

16. A computer readable storage device comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:

performing a first convolutional layer process associated with sequential downsampling of medical imaging data;

performing a second convolutional layer process associated with sequential upsampling of the medical imaging data, wherein a first convolutional layer of the first convolutional layer process corresponds to a last convolutional layer of the second convolutional layer process; and

predicting a medical condition associated with an anatomical region based on learned medical imaging output associated with the first convolutional layer process and the second convolutional layer process.

17. The computer readable storage device of claim 16, wherein the performing the first convolutional layer process comprises performing the first convolutional layer process based on a first convolutional layer filter that comprises a first size and a second convolutional layer filter that comprises a second size that is different than the first size.

18. The computer readable storage device of claim 17, wherein the performing the second convolutional layer process comprises performing the second convolutional layer process based on the first convolutional layer filter that comprises the first size and the second convolutional layer filter that comprises the second size.

19. The computer readable storage device of claim 16, wherein the performing the second convolutional layer process comprises performing the sequential upsampling of the medical imaging data in a reverse sampling sequence with respect to the sequential downsampling of the medical imaging data.

20. The computer readable storage device of claim 16, wherein the predicting the medical condition associated with the anatomical region comprises determining a classification for the medical condition associated with the anatomical region based on the learned medical imaging output.