Posterior image sampling using statistical learning model

• US 10,672,153 B2
• Filed: 11/13/2018
• Issued: 06/02/2020
• Est. Priority Date: 04/23/2018
• Status: Active Grant

• Current Assignee: Elekta AB
• Original Assignee: Elekta AB
• Inventors: Adler, Jonas Anders, Oktem, Ozan
• Primary Examiner(s): Ansari, Tahmina N
• Application Number:
  US16/189,480
• Publication Number:
  US 20190325620A1
• Time in Patent Office: 567 Days
• Field of Search: None
• CPC Class Codes: G06T 11/006 Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
• CPC Class Codes: G06T 11/008 Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2211/424 Iterative
• CPC Class Codes: G06T 2211/441 AI-based methods, deep learning or artificial neural networks
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215

Abstract

Image reconstruction can include using a statistical or machine learning, MAP estimator, or other reconstruction technique to produce a reconstructed image from acquired imaging data. A Conditional Generative Adversarial Network (CGAN) technique can be used to train a Generator, using a Discriminator, to generate posterior distribution sampled images that can be displayed or further processed such as to help provide uncertainty information about a mean reconstruction image. Such uncertainty information can be useful to help understand or even visually modify the mean reconstruction image. Similar techniques can be used in a segmentation use-case, instead of a reconstruction use case. The uncertainty information can also be useful for other post-processing techniques.

Claims

1. A method of image processing using image processor circuitry applying a trained statistical learning model, the method comprising:

obtaining imaging data of a human subject acquired at least in part using a medical imaging modality;

applying at least one of a reconstruction or a segmentation to the imaging data to generate at least one reconstructed or segmented initial image;

providing the initial image and an image randomness component to a generator neural network, the generator neural network applying a previously trained conditional generative statistical learning model;

generating, with the generator neural network, a plurality of posterior distribution simulated images from the initial image and the image randomness component;

analyzing the posterior distribution simulated images, to identify at least one indication of an image error associated with the initial image; and

providing the indication of the image error associated with the initial image for displaying or further processing.

2. The method of claim 1, wherein providing the indication of the image error associated with the initial image includes providing a point-wise indication of the image error associated with the initial image.

3. The method of claim 2, wherein providing the point-wise indication of the image error associated with the initial image includes displaying the posterior distribution simulated images as frames in a video.

4. The method of claim 2, wherein providing the point-wise indication of the image error associated with the initial image includes displaying a composite error image determined from the posterior distribution simulated images.

5. The method of claim 1, comprising, using the image processor circuitry, evaluating a feature at least in part using the posterior distribution simulated images.

6. The method of claim 5, wherein evaluating the feature includes determining how often the feature is present in the posterior distribution simulated images.

7. The method of claim 5, wherein evaluating the feature includes determining whether a spatiotemporal position of the feature is changing between the posterior distribution simulated images.

8. The method of claim 5, wherein evaluating the feature includes determining a spatiotemporal presence of the feature in the posterior distribution simulated images.

9. The method of claim 1, comprising modifying the initial image using the indication of the image error for display to a user.

10. The method of claim 9, wherein the initial image includes a mean reconstruction image, and wherein the modifying the initial image includes adding a variation to texture at least one region of the mean reconstruction image.

11. The method of claim 1, wherein the applying at least one of a reconstruction or a segmentation to the imaging data to generate at least one reconstructed or segmented initial image includes applying a reconstruction to the imaging data to generate at least one reconstructed initial image.

12. The method of claim 1, wherein applying a reconstruction or a segmentation to the imaging data to generate at least one reconstructed or segmented initial image includes applying a segmentation to the imaging data to generate at least one segmented initial image.

13. The method of claim 1 wherein the generator neural network applies a statistical learning model trained including by:

providing at least one reconstructed or segmented initial image of an object to a discriminator network and a generator network of a conditional generative adversarial network (GAN);

providing at least one corresponding deemed true image of the object to the discriminator network;

using the generator network to generate, from the initial image and from an image randomness component, a posterior distribution simulated image, conditioned on the initial image provided to the discriminator network; and

using the discriminator network to use the initial image to distinguish between the posterior distribution simulated images and the deemed true image to train the statistical-learning model to be applied by the generator network.

14. The method of claim 1, wherein presenting the indication of the image error includes displaying on an MR-LINAC workstation for use in radiation treatment planning or administration.

15. The method of claim 1, comprising updating, or receiving an update, of a radiation treatment protocol based at least in part upon the indication of the image error.

16. The method of claim 1, wherein the generator neural network implements a first convolutional neural network of a conditional generative adversarial network (GAN) that also includes a discriminator neural network that implements a second convolutional neural network.

17. The method of claim 1, wherein the generator neural network implements a variational auto-encoder (VAE).

18. A method of training image processor circuitry using statistical learning, the method comprising:

providing at least one reconstructed or segmented initial image of an anatomical object captured from a human subject to a discriminator neural network and a generator neural network;

providing at least one corresponding deemed true image of the anatomical object to the discriminator neural network;

using the generator neural network to generate, from the initial image and an image randomness component, posterior distribution simulated images, conditioned on the initial image, provided to the discriminator neural network; and

using the discriminator neural network and the initial image to distinguish between the posterior distribution simulated images and the deemed true image, causing training of a statistical learning model, the statistical learning model adapted to be implemented by the generator neural network in generative operations for generating posterior distribution simulated images;

wherein analysis of the posterior distribution simulated images generated from a subsequently obtained at least one reconstructed or segmented initial image enables identification of an image error associated with the subsequently obtained at least one reconstructed or segmented initial image.

19. The method of claim 18, comprising training the discriminator neural network using the posterior distribution simulated images and the deemed true image.

20. The method of claim 19, comprising training the discriminator neural network using a conditional generative adversarial network (CGAN).