Protocol independent image processing with adversarial networks

• US 10,624,558 B2
• Filed: 08/06/2018
• Issued: 04/21/2020
• Est. Priority Date: 08/10/2017
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Ceccaldi, Pascal, Odry, Benjamin L., Mailhe, Boris, Nadar, Mariappan S.
• Primary Examiner(s): Mizrahi, Diane D
• Application Number:
  US16/055,546
• Publication Number:
  US 20190046068A1
• Time in Patent Office: 624 Days
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 11/003 Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 9/002 using neural networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

Systems and methods are provided for generating a protocol independent image. A deep learning generative framework learns to recognize the boundaries and classification of tissues in an MRI image. The deep learning generative framework includes an encoder, a decoder, and a discriminator network. The encoder is trained using the discriminator network to generate a latent space that is invariant to protocol and the decoder is trained to generate the best output possible for brain and/or tissue extraction.

Claims

1. A method for generating domain independent magnetic resonance images in a magnetic resonance imaging system, the method comprising:

scanning a patient by the magnetic resonance imaging system to acquire magnetic resonance data;

inputting the magnetic resonance data to a machine learnt generator network trained to extract features from input magnetic resonance data and reconstruct domain independent images using the extracted features;

generating, by the machine learnt generator network, a domain independent magnetic resonance image from the input magnetic resonance data; and

displaying the domain independent magnetic resonance image,

wherein the machine learnt generator network comprises an encoder configured to generate a compact representation of the input magnetic resonance data and a decoder configured to reconstruct the domain independent image from the compact representation,

wherein the machine learnt generator network is trained using a loss function that is calculated as a combination of a first value, computed from a first loss function provided by the decoder and a second value, computed from a second loss function provided by a first adversarial learnt network trained to classify concatenated features from the compact representation as either from a first domain or a second domain.

2. The method of claim 1, wherein the second loss function is calculated as a function of a Wasserstein distance.

3. The method of claim 1, wherein the second loss function is calculated as a function of a Cramer distance.

4. The method of claim 1, wherein the first domain represents ground truth data.

5. The method of claim 1, wherein the machine learnt generator network is further trained using a second adversarial learnt network trained to classify generated domain independent images as generated by the machine learnt generator network or ground truth images.

6. The method of claim 1, wherein the domain independent magnetic resonance image is a segmented image.

7. The method of claim 6, wherein the segmented image comprises a segmented brain image including boundaries for at least white matter, grey matter, and cerebrospinal fluid.