AUTOMATED SEGMENTATION OF ORGAN CHAMBERS USING DEEP LEARNING METHODS FROM MEDICAL IMAGING

• US 20170109881A1
• Filed: 10/14/2016
• Published: 04/20/2017
• Est. Priority Date: 10/14/2015
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Avendi, Michael Rashidi, Jafarkhani, Hamid, Kheradvar, Arash
• Granted Patent: US 10,521,902 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20124 Active shape model [ASM]
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 2201/031 of internal organs

Abstract

Systems and methods are disclosed for automatically segmenting a heart chamber from medical images of a patient. The system may include one or more hardware processors configured to: obtain image data including at least a representation of the patient'"'"'s heart; obtain a region of interest from the image data; organize the region of interest into an input vector; apply the input vector through a trained graph; obtain an output vector representing a refined region of interest corresponding to the heart based on the application of the input vector through the trained graph; apply a deformable model on the obtained output vector representing the refined region of interest; and identify a segment of a heart chamber from the application of the deformable model on the obtained output vector.

Claims

1. A system for automatically segmenting a body chamber from medical images of a patient, the system comprising one or more hardware processors configured to:

obtain medical images comprising at least a representation of the body chamber inside the patient;

obtain a region of interest corresponding to the body chamber from each of the medical images;

organize the obtained region of interest into an input vector;

apply the input vector through a trained graph having at least one hidden layer; and

obtain an output vector representing a refined region of interest corresponding to the body chamber based on the application of the input vector through the trained graph.

2. The system according to claim 1, wherein the hardware processors are configured to apply a deformable model on the obtained output vector representing the region of interest.

3. The system according to claim 1, wherein the hardware processors are configured to identify a segment of a body chamber from an output of the trained graph.

4. The system according to claim 1, wherein the one or more hardware processors is/are configured to identify and segment a chamber of a heart.

5. The system according to claim 4, wherein the chamber of a heart is selected from the group consisting of a left ventricle, a right ventricle, a left atrium and a right atrium.

6. The system according to claim 1, wherein the medical images comprise magnetic resonance imaging (MRI), echocardiographic or CT scan data.

7. The system according to claim 1, wherein the system is configured to obtain filters using a sparse autoencoder (AE), which acts as a pre-training step.

8. The system according to claim 1, wherein the trained graph has two or more hidden layers.

9. The system according to claim 1, wherein the system is configured to utilize a training data set to initialize filters randomly to train a convolutional network.

10. The system according to claim 7, wherein the filters are convolved with the input medical images to obtain k convolved feature maps of size m₁×

m₁, computed as;

ZI[i,j]=k1=1ak2=1aFI[k1,k2]I[i+k1−

1,j+k2−

1]+b0[I], 

 

(1)

for 1≦

m₁, l=1, . . . , k, and m₁=m−

a+1; and

wherein p×

p non-overlapping regions in the convolved feature maps are computed as;

11. The system according to claim 1, wherein the one or more hardware processors is configured to align short-axis images of the heart chamber by performing contour alignment to reduce misalignment between the short-axis images.

12. The system according to claim 11, wherein center coordinates of the images are estimated using the following quadratic assumptions for curvature:

x_i=a₁i²+b₁i+c₁ 

 

(Equation

     17),

y_i=a₂i²+b₂i+c₁ 

 

(Equation

     18),

wherein a₁, b₁, c₁, a₂, b₂, c₂ are unknown parameters estimated based on minimizing the mean squared error as;

13. The system according to claim 1, wherein the system comprises one or more elements selected from the group consisting of an operating system, a data input port or reader, an output display device, a keyboard, a transceiver unit, a speaker, and an antenna for wireless communication with other with other devices.

14. The system according to claim 1, wherein the system is linked to an additional computing device and/or a network.

15. The system according to claim 1, wherein the memory comprises one or more of a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a hard disk drive, a floppy disk drive and a disk-based memory.

16. The system according to claim 1, wherein the system comprises a graphics processing unit (GPU) configured to accelerate computing.

17. A method of detecting whether or not a body chamber has an abnormal structure comprising:

providing a stack of short-axis images as input to the system according to claim 1,

operating the system to;

detect the body chamber in the images using deep convolutional networks trained to locate the body chamber,

infer a shape of the body chamber using a stacked auto-encoder trained to delineate the body chamber,

incorporate the inferred shape into a deformable model for segmentation, and

detecting whether or not the body chamber has an abnormal structure, wherein an abnormal structure is indicated by a body chamber clinical indicia that is different from a corresponding known standard clinical indicia for the body chamber.

18. The method according to claim 17, further comprising performing contour alignment to reduce misalignment between multiple short-axis medical images.

19. The method according to claim 17, wherein the clinical indicia is selected from the group consisting of:

a volume of the body chamber, an ejection fraction, a mass of the body chamber or a chamber'"'"'s wall thickness of the body chamber.

20. A method of detecting whether a body chamber exhibits an abnormal functional motion comprising:

providing a stack of short-axis medical images as input to the system according to claim 1,

operating the system to;

detect the body chamber in the images using deep convolutional networks trained to locate the body chamber,

infer a shape of a body chamber using a stacked auto-encoder trained to delineate the body chamber,

incorporate the inferred shape into a deformable model for segmentation, and

align stack images of the body chamber by performing contour alignment to reduce misalignment between the short-axis images; and

measuring clinical indicia at two time points, and

detecting whether the body chamber shows an abnormal functional motion, wherein an abnormal functional motion is indicated by an abnormal change in the clinical indicia.

21. The method according to claim 20, wherein the clinical indicia measured is selected from the group consisting of a volume of a heart chamber, an ejection fraction, a mass of a heart chamber and a wall thickness of a heart chamber.

22. A method for automatically segmenting a body chamber from medical images of a patient, the method comprising one or more processes to:

obtain image data representing the medical images comprising at least a representation of a body chamber inside the body;

obtain a region of interest corresponding to the body chamber such as but not limited to the heart from the image data;

organize the obtained region of interest into an input vector;

apply the input vector through a trained graph having at least one hidden layer;

obtain an output vector representing a refined region of interest corresponding to the body chamber based on the application of the input vector through the trained graph;

apply a deformable model on the obtained output vector representing the refined region of interest; and

identify a segment of a body chamber from the application of the deformable model on the obtained output vector.