Image Correction Using A Deep Generative Machine-Learning Model

• US 20170372193A1
• Filed: 05/16/2017
• Published: 12/28/2017
• Est. Priority Date: 06/23/2016
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Mailhe, Boris, Cetingul, Hasan Ertan, Odry, Benjamin L., Chen, Xiao, Nadar, Mariappan S.
• Granted Patent: US 10,387,765 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection

Abstract

For correction of an image from an imaging system, a deep-learnt generative model is used as a regularlizer in an inverse solution with a physics model of the degradation behavior of the imaging system. The prior model is based on the generative model, allowing for correction of an image without application specific balancing. The generative model is trained from good images, so difficulty gathering problem-specific training data may be avoided or reduced.

Claims

1. A method for correction of an image from a medical scanner, the method comprising:

acquiring, by the medical scanner, the image representing a patient, the image having a level of artifact due to the acquisition by the medical scanner;

determining, by a machine, a probability of artifact abnormality for the image with a deep generative machine-learnt model;

minimizing, by the machine, the level of the artifact in the image, the minimizing being a function of a physics model and the probability, the physics model including a characteristic specific to the medical scanner; and

transmitting the image output from the minimizing, the image being of the patient and from the medical scanner with the level of the artifact minimized.

2. The method of claim 1 wherein acquiring comprises computed tomography, magnetic resonance, ultrasound, positron emission tomography, or single photon emission computed tomography image.

3. The method of claim 1 wherein acquiring comprises representing the output as a two-dimensional representation of pixels or a three-dimensional set of voxels as the image.

4. The method of claim 1 wherein acquiring comprises acquiring with corruption by noise artifact, blur artifact, inpainting artifact, reconstruction artifact, or combinations thereof.

5. The method of claim 1 wherein determining comprises determining with the deep generative machine-learnt model learnt with only training images having a quality above a threshold.

6. The method of claim 5 wherein determining comprises determining the probability to maximize the log-likelihood of the good training images matching the deep generative machine-learnt model.

7. The method of claim 1 wherein determining comprises determining with the deep generative machine-learnt model learnt with multi-objective training including training images labeled as having quality below a threshold.

8. The method of claim 7 wherein determining comprises determining the probability to maximize the posterior likelihood of the corrected images obtained from the bad training images.

9. The method of claim 1 wherein determining comprises determining with the deep generative machine-learnt model learnt with multi-objective training including training pairs of low-quality images with corresponding target output image of the artifact correction.

10. The method of claim 9 wherein determining comprises determining the probability to minimize a difference between the target output image and the application of the correction applied to the low-quality images.

11. The method of claim 1 wherein determining comprises determining with the deep-generative machine-learnt model learnt from training images with different types of artifacts including the artifact.

12. The method of claim 1 wherein determining comprises determining with a machine-learnt discriminative network trained from the deep generative machine-learnt model.

13. The method of claim 1 wherein minimizing comprises inversely solving with the probability as a regularizer term.

14. The method of claim 1 wherein minimizing comprises minimizing by gradient descent with the physics model in a first term and a gradient of the probability in a second term, the physics model including a distribution of the characteristic specific to the medical scanner.

15. The method of claim 1 wherein minimizing further comprises determining a gradient direction for a next iteration in the minimizing, the gradient direction determined from the deep generative machine-learnt model.

16. The method of claim 1 further comprising providing a step size, number of iterations, preconditioner, or combinations thereof in the minimizing with a machine-learnt classifier applied to the image.

17. The method of claim 1 wherein transmitting comprises transmitting the image to a display.

18. A method for reducing distortion in an image from an imaging system, the method comprising:

optimizing the image of an object to have less of the distortion, the optimizing being with a gradient-based optimization including a regularizer from a log-likelihood output by a machine-learnt generative model; and

displaying the image as optimized.

19. The method of claim 18 wherein optimizing comprises optimizing with a physics model encoding a transform of the imaging system.

20. The method of claim 18 wherein optimizing comprises optimizing with the machine-learnt generative model being based on training images having a quality above a threshold and being based on a supervised sub-set of training imaging having known ground truth relative to a type of the distortion.

21. A method for correction of a first image from an imaging system, the method comprising:

acquiring, by the imaging system, the first image having a distortion due to the acquisition by the imaging system;

determining, by a machine, a likelihood of the first image having the distortion with a deep generative machine-learnt model;

solving for a corrected image from the first image, the solving using the likelihood and a transform for the imaging system; and

transmitting the corrected image.

22. The method of claim 21 wherein determining comprises determining with the deep generative machine-learnt model trained with multi-objective training.