Multi-scale deep reinforcement machine learning for N-dimensional segmentation in medical imaging

• US 10,643,331 B2
• Filed: 06/22/2018
• Issued: 05/05/2020
• Est. Priority Date: 05/03/2017
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Ghesu, Florin Cristian, Georgescu, Bogdan, Comaniciu, Dorin
• Primary Examiner(s): Mistry, Oneal R
• Application Number:
  US16/015,231
• Publication Number:
  US 20180322637A1
• Time in Patent Office: 683 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20116 Active contour; Active surface; Snakes
• CPC Class Codes: G06T 2207/20124 Active shape model [ASM]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/60 for the operation of medical equipment or devices
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Multi-scale deep reinforcement learning generates a multi-scale deep reinforcement model for multi-dimensional (e.g., 3D) segmentation of an object. In this context, segmentation is formulated as learning an image-driven policy for shape evolution that converges to the object boundary. The segmentation is treated as a reinforcement learning problem, and scale-space theory is used to enable robust and efficient multi-scale shape estimation. By learning an iterative strategy to find the segmentation, the learning challenges of end-to-end regression systems may be addressed.

Claims

1. A method for three-dimensional segmentation based on machine learning in a medical imaging system, the method comprising:

scanning, by a magnetic resonance, computed tomography, x-ray, or ultrasound imaging system, a patient, the scanning providing a medical dataset representing a three-dimensional region of a patient, the medical dataset comprising magnetic resonance, computed tomography, x-ray, or ultrasound data;

applying, by a machine, the medical dataset to a multi-scale deep reinforcement machine-learnt model, the multi-scale deep reinforcement machine-learned model trained with multi-scale deep reinforcement learning to segment boundaries of a three-dimensional object from the medical dataset, the multi-scale deep reinforcement machine-learnt model including a machine-learnt policy of a sequence of actions for shape evolution over iterative refinements of the boundaries of the three-dimensional object, the machine-learnt policy trained to select each of the actions from possibilities given a state of fitting at each of the iterative refinements of shape representation parameters, the sequence resulting from the machine-learnt policy training based on rewards, and the boundaries and refinements described using the shape representation parameters;

rendering, by a renderer, an image of the three-dimensional object based on the boundaries determined with the policy; and

displaying the image on a display device.

2. The method of claim 1 further comprising loading patient information other than the medical dataset representing the three-dimensional region of the patient, and wherein applying comprises applying the medical dataset and the patient information.

3. The method of claim 1 wherein applying comprises applying with the machine-learnt policy comprising refinements of the boundaries over scale in different resolutions and location.

4. The method of claim 1 wherein applying comprises applying with the multi-scale deep reinforcement machine-learnt model comprising an action, of the sequence of actions, in space for each of the iterative refinements and estimation of locations of the three-dimensional object based on the action for each of the iterative refinements.

5. The method of claim 1 wherein applying comprises determining different shapes of the three-dimensional object in the different iterative refinements.

6. The method of claim 1 wherein applying comprises applying to the multi-scale deep reinforcement machine-learnt model trained with deep Q-learning.

7. The method of claim 6 wherein applying comprises applying the multi-scale deep reinforcement machine-learnt model trained with a Markov decision process including a compact system-state of the medical dataset and the shape representation parameters parameterizing statistical shape-modeling, front propagation modeling, or voxel mask modeling.

8. The method of claim 6 wherein applying comprises applying the multi-scale deep reinforcement machine-learnt model trained with a Markov decision process including an action space controlling the shape evolution.

9. The method of claim 6 wherein applying comprises applying the multi-scale deep reinforcement machine-learnt model trained with a Markov decision process including feature extraction for the three-dimensional object and effective finite-horizon policy learning of the machine-learnt policy.

10. The method of claim 1 wherein applying comprises applying with the iterative refinements comprising interaction between an intelligent agent forming the machine-learnt policy for shape evolution and statistical shape-modeling, front propagation modeling, or voxel mask modeling of the shape representation parameters, the intelligent agent passing the shape evolution for each iteration to the statistical shape-modeling, front propagation modeling, or voxel mask modeling, and the statistical shape-modeling, front propagation modeling, or a voxel mask model passing a current shape descriptor to the intelligent agent for each iteration.

11. The method of claim 1 wherein applying comprises applying with the boundaries and refinements described using statistical shape modeling for the shape representation parameters and wherein the shape evolution comprises change in one or more shape parameters of a projected shape-space of the statistical shape modeling.

12. The method of claim 1 wherein applying comprises applying with the boundaries and refinements described using front propagation modeling for the shape representation parameters and wherein the shape evolution comprises change in a speed of particles in a hypersurface of the front propagation modeling.

13. The method of claim 1 wherein applying comprises applying with the boundaries and refinements described using voxel mask modeling for the shape representation parameters and wherein the shape evolution comprises change in state per voxel of the medical dataset of the voxel mask modeling.

14. The method of claim 1 wherein applying comprises applying with the multi-scale deep reinforcement machine-learnt model trained with natural evolution strategies to explore a parameter space of the shape representation parameters based on statistical shape-modeling, front propagation modeling, or voxel mask modeling.

15. The method of claim 1 wherein rendering the image comprises rendering from the medical dataset with the segmented boundaries highlighted.

16. A method for multi-dimensional segmentation based on machine learning in a medical imaging system, the method comprising:

scanning, by a magnetic resonance, computed tomography, x-ray, or ultrasound imaging system, a patient, the scanning providing a medical dataset representing a three-dimensional region of a patient, the medical dataset comprising magnetic resonance, computed tomography, x-ray, or ultrasound data;

applying, by a machine, the medical dataset to a multi-scale, deep reinforcement machine-learnt model, the multi-scale, deep reinforcement machine-learned model trained with multi-scale, deep reinforcement learning to segment boundaries of a multi-dimensional object from the medical dataset, the multi-scale, deep reinforcement machine-learnt model including a machine-learnt policy for shape evolution over iterative refinements of the boundaries in scale at different resolutions of the medical dataset and in location, the multi-scale deep reinforcement machine-learnt model trained to explore a parameter space; and

generating, by a graphics processor, an image of the multi-dimensional object based on the boundaries determined with the policy.

17. The method of claim 16 wherein applying comprises applying with the machine-learnt policy for the shape evolution uses a shape descriptor based on a level-set, statistical shape model, or voxel mask with the refinements being of a change in speed, change of a shape space parameter, or change in state, respectively.

18. The method of claim 16 wherein applying comprises applying with the shape evolution over the iterative refinements in different scales corresponding to the different resolutions.

19. The method of claim 16 wherein applying comprises applying with the shape evolution over the iterative refinements having a control output by the machine-learnt policy to a shape descriptor model and a shape descriptor output by the shape descriptor model to the machine-learnt policy based on the control.

20. The method of claim 16 wherein applying comprises applying with the multi-scale, deep reinforcement machine-learnt model trained for the sequence of actions in how to segment.