Multi-scale deep reinforcement machine learning for N-dimensional segmentation in medical imaging

• US 10,032,281 B1
• Filed: 07/27/2017
• Issued: 07/24/2018
• Est. Priority Date: 05/03/2017
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Ghesu, Florin Cristian, Georgescu, Bogdan, Comaniciu, Dorin
• Primary Examiner(s): Mistry, Oneal R
• Application Number:
  US15/661,675
• Time in Patent Office: 362 Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20116 Active contour; Active surface; Snakes
• CPC Class Codes: G06T 2207/20124 Active shape model [ASM]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/60 for the operation of medical equipment or devices
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Multi-scale deep reinforcement learning generates a multi-scale deep reinforcement model for multi-dimensional (e.g., 3D) segmentation of an object. In this context, segmentation is formulated as learning an image-driven policy for shape evolution that converges to the object boundary. The segmentation is treated as a reinforcement learning problem, and scale-space theory is used to enable robust and efficient multi-scale shape estimation. By learning an iterative strategy to find the segmentation, the learning challenges of end-to-end regression systems may be addressed.

Claims

1. A method for three-dimensional segmentation based on machine learning in a medical imaging system, the method comprising:

scanning, by a medical scanner, a patient, the medical scanner comprising a magnetic resonance, computed tomography, x-ray, or ultrasound imaging system;

loading, from memory, a medical dataset representing a three-dimensional region of the patient, the medical dataset being from the scanning by the medical scanner, the medical dataset comprising magnetic resonance, computed tomography, x-ray, or ultrasound data;

applying, by a machine, the medical dataset to a multi-scale deep reinforcement machine-learnt model, the multi-scale deep reinforcement machine-learned model trained with multi-scale deep reinforcement learning to segment boundaries of a three-dimensional object from the medical dataset, the multi-scale deep reinforcement machine-learnt model including a machine-learnt policy of a sequence of actions for shape evolution over iterative refinements of the boundaries of the object, the machine-learnt policy trained to select each of the actions from possibilities given a state of fitting at each of the iterative refinements, the actions including actions in different resolutions of the medical dataset, the sequence resulting from the machine-learnt policy training based on maximizing rewards, the boundaries and refinements described using statistical shape-modeling, front propagation modeling, or voxel mask modeling, the multi-scale deep reinforcement machine-learnt model trained with natural evolution strategies to explore a parameter space based on the statistical shape-modeling, front propagation modeling, or voxel mask modeling;

rendering, by a renderer, an image of the three-dimensional object based on the boundaries determined with the policy; and

displaying the image on a display device.

2. The method of claim 1 further comprising loading patient information other than the medical dataset representing the three-dimensional region of the patient, and wherein applying comprises applying the medical dataset and the patient information.

3. The method of claim 1 wherein applying comprises applying with the machine-learnt policy comprising refinements of the boundaries over scale in the different resolutions and location.

4. The method of claim 1 wherein applying comprises applying with the multi-scale deep reinforcement machine-learnt model comprising an action, of the sequence of actions, in space for each of the iterative refinements and estimation of locations of the three-dimensional object based on the action for each of the iterative refinements.

5. The method of claim 1 wherein applying comprises determining different shapes of the three-dimensional object in the different iterative refinements.

6. The method of claim 1 wherein applying comprises applying to the multi-scale deep reinforcement machine-learnt model trained with deep Q-learning.

7. The method of claim 6 wherein applying comprises applying the multi-scale deep reinforcement machine-learnt model trained with the Markov decision process including a compact system-state of the medical dataset and a shape representation comprising the statistical shape-modeling, the front propagation modeling, or the voxel mask modeling.

8. The method of claim 6 wherein applying comprises applying the multi-scale deep reinforcement machine-learnt model trained with the Markov decision process including an action space controlling the shape evolution.

9. The method of claim 6 wherein applying comprises applying the multi-scale deep reinforcement machine-learnt model trained with the Markov decision process including feature extraction for the three-dimensional object and effective finite-horizon policy learning of the machine-learnt policy.

10. The method of claim 1 wherein applying comprises applying with the iterative refinements comprising interaction between an intelligent agent forming the machine-learnt policy for shape evolution and the statistical shape-modeling, front propagation modeling, or voxel mask modeling, the intelligent agent passing the shape evolution for each iteration to the statistical shape-modeling, front propagation modeling, or voxel mask modeling, and the statistical shape-modeling, front propagation modeling, or a voxel mask model passing a current shape descriptor to the intelligent agent for each iteration.

11. The method of claim 1 wherein applying comprises applying with the boundaries and refinements described using the statistical shape modeling and wherein the shape evolution comprises change in one or more shape parameters of a projected shape-space of the statistical shape modeling.

12. The method of claim 1 wherein applying comprises applying with the boundaries and refinements described using the front propagation modeling and wherein the shape evolution comprises change in a speed of particles in a hypersurface of the front propagation modeling.

13. The method of claim 1 wherein applying comprises applying with the boundaries and refinements described using the voxel mask modeling and wherein the shape evolution comprises change in state per voxel of the medical dataset of the voxel mask modeling.

14. The method of claim 1 wherein rendering the image comprises rendering from the medical dataset with the segmented boundaries highlighted.

15. A method for multi-dimensional segmentation based on machine learning in a medical imaging system, the method comprising:

scanning, by a medical scanner, a patient, the medical scanner comprising a magnetic resonance, computed tomography, x-ray, or ultrasound imaging system;

loading, from memory, a medical dataset representing a three-dimensional region of the patient, the medical dataset being from the scanning by the medical scanner, the medical dataset comprising magnetic resonance, computed tomography, x-ray, or ultrasound data;

applying, by a machine, the medical dataset to a multi-scale, deep reinforcement machine-learnt model, the multi-scale, deep reinforcement machine-learned model trained with multi-scale, deep reinforcement learning to segment boundaries of a multi-dimensional object from the medical dataset, the multi-scale, deep reinforcement machine-learnt model including a machine-learnt policy of a sequence of actions for shape evolution over iterative refinements of the boundaries in scale at different resolutions of the medical dataset and in location, the multi-scale deep reinforcement machine-learnt model trained with natural evolution strategies to explore a parameter space; and

generating, by a graphics processor on a display device, an image of the multi-dimensional object based on the boundaries determined with the policy.

16. The method of claim 15 wherein applying comprises applying with the machine-learnt policy for the shape evolution uses a shape descriptor based on a level-set, statistical shape model, or voxel mask with the refinements being of a change in speed, change of a shape space parameter, or change in state, respectively.

17. The method of claim 15 wherein applying comprises applying with the shape evolution over the iterative refinements in different scales corresponding to the different resolutions.

18. The method of claim 15 wherein applying comprises applying with the shape evolution over the iterative refinements having a control output by the machine-learnt policy to a shape descriptor model and a shape descriptor output by the shape descriptor model to the machine-learnt policy based on the control.

19. The method of claim 15 wherein applying comprises applying with the multi-scale, deep reinforcement machine-learnt model trained for the sequence of actions in how to segment.

20. A method for three-dimensional segmentation based on machine learning in a medical imaging system, the method comprising:

scanning, by a medical scanner, a patient, the medical scanner comprising a magnetic resonance, computed tomography, x-ray, or ultrasound imaging system;

loading, from memory, a medical dataset representing a three-dimensional region of the patient, the medical dataset being from the scanning by the medical scanner, the medical dataset comprising magnetic resonance, computed tomography, x-ray, or ultrasound data;

applying, by a machine, the medical dataset to a multi-scale deep reinforcement machine-learnt model, the multi-scale deep reinforcement machine-learned model trained with multi-scale deep reinforcement learning to segment boundaries of a three-dimensional object from the medical dataset, the multi-scale deep reinforcement machine-learnt model including a machine-learnt policy of a sequence of actions for shape evolution over iterative refinements of the boundaries of the object, including actions in different resolutions of the medical dataset, the boundaries and refinements described using front propagation modeling, or voxel mask modeling, wherein applying comprises one of;

(a) applying with the boundaries and refinements described using the front propagation modeling and wherein the shape evolution comprises change in a speed of particles in a hypersurface of the front propagation modeling, or

(b) applying with the boundaries and refinements described using the voxel mask modeling and wherein the shape evolution comprises change in state per voxel of the medical dataset of the voxel mask modeling;

rendering, by a renderer, an image of the three-dimensional object based on the boundaries determined with the policy; and

displaying the image on a display device.