AUTOMATIC METHOD AND SYSTEM FOR VESSEL REFINE SEGMENTATION IN BIOMEDICAL IMAGES USING TREE STRUCTURE BASED DEEP LEARNING MODEL

• US 20190325579A1
• Filed: 04/23/2019
• Published: 10/24/2019
• Est. Priority Date: 04/24/2018
• Status: Active Grant

• Current Assignee: Shenzhen Keya Medical Technology Corporation
• Original Assignee: Shenzhen Keya Medical Technology Corporation
• Inventors: Wang, Xin, Yin, Youbing, Bai, Junjie, Lu, Yi, Song, Qi
• Granted Patent: US 10,430,949 B1
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images

Abstract

Embodiments of the disclosure provide systems and methods for segmenting a biomedical image including at least one tree structure object. The system includes a communication interface configured to receive the biomedical image and a learning model. The biomedical image is acquired by an image acquisition device. The system further includes at least one processor configured to extract a plurality of image patches from the biomedical image and apply the learning model to the plurality of image patches to segment the biomedical image. The learning model includes a convolutional network configured to process the plurality of image patches to construct respective feature maps and a tree structure network configured to process the feature maps collectively to obtain a segmentation mask for the tree structure object. The tree structure network models a spatial constraint of the plurality of image patches.

Claims

1. A system for segmenting a biomedical image including at least one tree structure object, comprising:

a communication interface configured to receive the biomedical image and a learning model, the biomedical image being acquired by an image acquisition device; and

at least one processor, configured to;

extract a plurality of image patches from the biomedical image; and

apply the learning model to the plurality of image patches to segment the biomedical image, wherein the learning model includes a convolutional network configured to process the plurality of image patches to construct respective feature maps and a tree structure network configured to process the feature maps collectively to obtain a segmentation mask for the tree structure object, wherein the tree structure network models a spatial constraint of the plurality of image patches.

2. The system of claim 1, wherein to extract the plurality of image patches, the at least one processor is further configured to:

determine a centerline of the tree structure object in the biomedical image; and

extract the plurality of image patches along the centerline.

3. The system of claim 1, wherein the convolutional network includes a plurality of fully convolutional networks each configured to process an image patch.

4. The system of claim 3, wherein the plurality of fully convolutional networks process the plurality of image patches in parallel.

5. The system of claim 1, wherein the tree structure network is a tree structure convolution recurrent neural network.

6. The system of claim 1, wherein the spatial constraint is determined based on a tree structure of the image patches.

7. The system of claim 1, wherein the spatial constraint is determined based on a spatial relationship between two neighboring image patches among the plurality of image patches.

8. The system of claim 1, wherein the convolutional network and the tree structure network are trained jointly using a gradient descent based method.

9. The system of claim 1, wherein the tree structure object is a blood vessel.

10. The system of claim 1, wherein the biomedical image is a 3D image including a plurality of 2D image slices, wherein the plurality of image patches are extracted across the plurality of 2D image slices, wherein the spatial constraint is determined based on a spatial relationship of the plurality of 2D image slices.

11. A method for segmenting a biomedical image including at least one tree structure object, comprising:

receiving the biomedical image and a learning model, the biomedical image being acquired by an image acquisition device;

extracting, by at least one processor, a plurality of image patches from the biomedical image; and

applying, by the at least one processor, the learning model to the plurality of image patches to segment the biomedical image, wherein the learning model includes a convolutional network configured to process the plurality of image patches to construct respective feature maps and a tree structure network configured to process the feature maps collectively to obtain a segmentation mask for the tree structure object, wherein the tree structure network models a spatial constraint of the plurality of image patches.

12. The method of claim 11, wherein extracting the plurality of image patches further includes:

determining a centerline of the tree structure object in the biomedical image; and

extracting the plurality of image patches along the centerline.

13. The method of claim 11, wherein the convolutional network includes a plurality of fully convolutional networks each configured to process an image patch.

14. The method of claim 13, wherein the plurality of fully convolutional networks process the plurality of image patches in parallel.

15. The method of claim 11, wherein the tree structure network is a tree structure convolution recurrent neural network.

16. The method of claim 11, wherein the spatial constraint is determined based on a tree structure of the image patches.

17. The method of claim 11, wherein the spatial constraint is determined based on a spatial relationship between two neighboring image patches among the plurality of image patches.

18. The method of claim 11, wherein the convolutional network and the tree structure network are trained jointly using a gradient descent based method.

19. The method of claim 11, wherein the tree structure object is a blood vessel.

20. A non-transitory computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by at least one processor, performs a method for segmenting a biomedical image including at least one tree structure object, the method comprising:

receiving the biomedical image and a learning model, the biomedical image being acquired by an image acquisition device;

extracting a plurality of image patches from the biomedical image; and

applying the learning model to the plurality of image patches to segment the biomedical image, wherein the learning model includes a convolutional network configured to process the plurality of image patches to construct respective feature maps and a tree structure network configured to process the feature maps collectively to obtain a segmentation mask for the tree structure object, wherein the tree structure network models a spatial constraint of the plurality of image patches.