PROTOCOL INDEPENDENT IMAGE PROCESSING WITH ADVERSARIAL NETWORKS

• US 20190046068A1
• Filed: 08/06/2018
• Published: 02/14/2019
• Est. Priority Date: 08/10/2017
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Ceccaldi, Pascal, Odry, Benjamin L., Mailhe, Boris, Nadar, Mariappan S.
• Granted Patent: US 10,624,558 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 11/003 Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 9/002 using neural networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

Systems and methods are provided for generating a protocol independent image. A deep learning generative framework learns to recognize the boundaries and classification of tissues in an MRI image. The deep learning generative framework includes an encoder, a decoder, and a discriminator network. The encoder is trained using the discriminator network to generate a latent space that is invariant to protocol and the decoder is trained to generate the best output possible for brain and/or tissue extraction.

Claims

1. A method for generating domain independent magnetic resonance images in a magnetic resonance imaging system, the method comprising:

scanning a patient by the magnetic resonance imaging system to acquire magnetic resonance data;

inputting the magnetic resonance data to a machine learnt generator network trained to extract features from input magnetic resonance data and reconstruct domain independent images using the extracted features;

generating, by the machine learnt generator network, a domain independent magnetic resonance image from the input magnetic resonance data; and

displaying the domain independent magnetic resonance image.

2. The method of claim 1, wherein the machine learnt generator network comprises an encoder configured to generate a compact representation of the input magnetic resonance data and a decoder configured to reconstruct the domain independent image from the compact representation.

3. The method of claim 2, wherein the machine learnt generator network is trained using a loss function that is calculated as a combination of a first value, computed from a first loss function provided by the decoder and a second value, computed from a second loss function provided by a first adversarial learnt network trained to classify concatenated features from the compact representation as either from a first domain or a second domain.

4. The method of claim 3, wherein the second loss function is calculated as a function of a Wasserstein distance.

5. The method of claim 3, wherein the second loss function is calculated as a function of a Cramer distance.

6. The method of claim 3, wherein the first domain represents ground truth data.

7. The method of claim 3, wherein the machine learnt generator network is further trained using a second adversarial learnt network trained to classify generated domain independent images as generated by the machine learnt generator network or ground truth images.

8. The method of claim 1, wherein the domain independent magnetic resonance image is a segmented image.

9. The method of claim 8, wherein the segmented image comprises a segmented brain image including boundaries for at least white matter, grey matter, and cerebrospinal fluid.

10. A method for training a generator neural network to output segmented MR images, the method comprising:

inputting, into an encoder network of the generator neural network, first data acquired from an MRI system, wherein the encoder network is configured to generate a latent space that represents a compact version of the first data;

inputting, into a decoder network of the generator neural network, the latent space, wherein the decoder network is configured to generate a segmented image;

calculating, by the decoder network, a first value based on a comparison of the segmented image and a ground truth segmented image;

inputting, into a discriminator network the latent space, wherein the discriminator network is configured to classify the latent space as acquired from a first domain or second domain;

calculating, by the discriminator network, a second value based on a second loss function for the classification;

adjusting the encoder network as a function of the first value and second value; and

repeating sequentially inputting, inputting, calculating, inputting, calculating, and adjusting until a training loss converges.

11. The method of claim 10, wherein the second value is calculated as a function of a Wasserstein distance.

12. The method of claim 10, wherein the second value is calculated as a function of a Cramer distance.

13. The method of claim 10, wherein the first domain represents ground truth data.

14. The method of claim 10, wherein the generator neural network is further trained using a second adversarial learnt network trained to classify generated domain independent images as generated by the generator neural network or ground truth images.

15. The method of claim 10, wherein the segmented MR images comprise segmented brain images including boundaries for at least white matter, grey matter, and cerebrospinal fluid.

16. The method of claim 10, wherein the latent space comprises compact feature data generated by the encoder network.

17. A system for generating domain independent magnetic resonance images, the system comprising:

a magnetic resonance system configured to acquire magnetic resonance data for a patient;

a generator network configured with a discriminator network and a critic function to generate domain independent magnetic resonance image data;

a memory configured to store the generator network as trained; and

a display configured to display the domain independent magnetic resonance image data from the generator network.

18. The system of claim 17, wherein the generator network comprises an encoder configured to generate a compact representation of the input magnetic resonance data and a decoder configured to reconstruct the domain independent image from the compact representation.

19. The system of claim 18, wherein the generator network is trained using a loss function that is calculated as a combination of a first value from a first loss function provided by the decoder and a second value from a second loss function provided by an adversarial learnt network trained to classify concatenated features from the compact representation as either from a first domain or a second domain.

20. The system of claim 17, the domain independent magnetic resonance image is a segmented image, wherein the segmented image comprises a segmented brain image including boundaries for at least white matter, grey matter, and cerebrospinal fluid.