Tomographic image reconstruction via machine learning

• US 10,970,887 B2
• Filed: 06/26/2017
• Issued: 04/06/2021
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Rensselaer Polytechnic Institute
• Original Assignee: Rensselaer Polytechnic Institute
• Inventors: Wang, Ge, Cong, Wenxiang, Yang, Qingsong
• Primary Examiner(s): Nguyen, Khai M
• Application Number:
  US16/312,704
• Publication Number:
  US 20190325621A1
• Time in Patent Office: 1,380 Days
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 6/5205 involving processing of raw data to produce diagnostic data
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 11/006 Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
• CPC Class Codes: G06T 11/008 Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 2211/421 Filtered back projection [FBP]
• CPC Class Codes: G06T 2211/424 Iterative

Abstract

Tomographic/tomosynthetic image reconstruction systems and methods in the framework of machine learning, such as deep learning, are provided. A machine learning algorithm can be used to obtain an improved tomographic image from raw data, processed data, or a preliminarily reconstructed intermediate image for biomedical imaging or any other imaging purpose. In certain cases, a single, conventional, non-deep-learning algorithm can be used on raw imaging data to obtain an initial image, and then a deep learning algorithm can be used on the initial image to obtain a final reconstructed image. All machine learning methods and systems for tomographic image reconstruction are covered, except for use of a single shallow network (three layers or less) for image reconstruction.

Claims

1. A method of reconstructing an image from tomographic data obtained by an imaging process, the method comprising:

performing at least one algorithm step on a raw data set or an intermediate data set of the tomographic data to obtain a final reconstructed image, wherein performing the at least one algorithm step comprises;

performing at least one conventional, non-deep-learning algorithm on a raw data of the tomographic data to obtain an intermediate data set of an initial reconstructed image; and

performing a deep learning algorithm on the intermediate data set to obtain the final reconstructed image.

2. The method according to claim 1, wherein performing at least one algorithm comprises performing a deep learning algorithm directly on the raw data set to obtain the final reconstructed image.

3. The method according to claim 1, wherein the deep learning algorithm is performed by a deep network.

4. The method according to claim 3, wherein the deep network is a deep neural network, and further comprising training the deep network with a training set of final images, prior to performing the deep learning algorithm.

5. The method according to claim 4, wherein training the deep network comprises performing on the deep network:

at least one fine-tuning technique, at least one feature selection technique, or both.

6. The method according to claim 1, wherein the deep learning algorithm is performed by a convolutional neural network (CNN).

7. The method according to claim 1, wherein raw data is obtained by computed tomography (CT), magnetic resonance imaging (MRI), single-photon emission computed tomography (SPECT), or positron emission tomography (PET).

8. The method according to claim 1, wherein performing at least one algorithm comprises performing a deep learning algorithm to complete a sinogram based on the tomographic data.

9. The method according to claim 1, wherein the raw data includes at least one metal artifact and the reconstructed image includes metal artifact reduction (MAR) compared to the raw data.

10. The method according to claim 1, wherein the deep learning algorithm is performed by a deep neural network, the deep neural network being AlexNet, ResNet, or GoogleNet.

11. The method according to claim 1, wherein the raw data comprises a CT image of one or more lung nodules.

12. The method according to claim 1, wherein the raw data comprises a low-dose CT image.

13. The method according to claim 1, wherein the deep learning algorithm reduces noise of the tomographic data such that the final reconstructed image has less noise than does the tomographic data.

14. The method according to claim 1, wherein the at least one conventional, non-deep-learning algorithm comprises at least one selected from the group of:

a normalization-based metal artifact reduction (NMAR) algorithm;

a filtered back projection (FBP) algorithm;

a model-based image reconstruction (MBIR) algorithm;

a block-matching 3D (BM3D) algorithm;

a total variation (TV) algorithm;

a K-SVD algorithm; and

an adaptive-steepest-descent (ASD) projection onto convex sets (POCS) (ASD-POCS) algorithm.

15. A system for reconstructing an image from raw data obtained by a medical imaging process, the system comprising:

a subsystem for obtaining tomographic imaging data;

at least one processor; and

a machine-readable medium, in operable communication with the subsystem for obtaining tomographic imaging data and the at least one processor, having machine-executable instructions stored thereon that, when executed by the at least one processor, perform at least one algorithm step on a raw data set or an intermediate data set of the tomographic data to obtain a final reconstructed image;

wherein the at least one algorithm step comprises;

performing at least one conventional, non-deep-learning algorithm on a raw data of the tomographic data to obtain an intermediate data set of an initial reconstructed image; and

performing a deep learning algorithm on the intermediate data set to obtain the final reconstructed image.

16. The system according to claim 15, wherein the deep learning algorithm is performed by a deep network.

17. The system according to claim 16, wherein the deep network is a deep neural network, and further comprising training the deep network with a training set of final images, prior to performing the deep learning algorithm.

18. The system according to claim 17, wherein training the deep network comprises performing on the deep network:

at least one fine-tuning technique, at least one feature selection technique, or both.

19. The system according to claim 15, wherein the deep learning algorithm is performed by a convolutional neural network (CNN).

20. The system according to claim 15, wherein the at least one conventional, non-deep-learning algorithm comprises at least one selected from the group of:

a normalization-based metal artifact reduction (NMAR) algorithm;

a filtered back projection (FBP) algorithm;

a model-based image reconstruction (MBIR) algorithm;

a block-matching 3D (BM3D) algorithm;

a total variation (TV) algorithm;

a K-SVD algorithm; and

an adaptive-steepest-descent (ASD) projection onto convex sets (POCS) (ASD-POCS) algorithm.