APPARATUS AND METHOD FOR VISUALIZING ANATOMICAL ELEMENTS IN A MEDICAL IMAGE

• US 20150265251A1
• Filed: 03/17/2015
• Published: 09/24/2015
• Est. Priority Date: 03/18/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Cho, Baek Hwan, Seong, Yeong Kyeong, Kim, Ye Hoon, Kim, Ha Young, Jeon, Joo Hyuk
• Granted Patent: US 10,383,602 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 8/0825 for diagnosis of the breast, e.g. mammography
• CPC Class Codes: A61B 8/085 for locating body or organic structures, e.g. tumours, calculi, blood vessels, nodules
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5215 involving processing of medical diagnostic data
• CPC Class Codes: A61B 8/5292 using additional data, e.g. patient information, image labeling, acquisition parameters
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/20128 Atlas-based segmentation
• CPC Class Codes: G06T 2207/30068 Mammography; Breast
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling

Abstract

A method of visualizing anatomical elements in a medical includes receiving a medical image; detecting a plurality of anatomical elements from the medical image; verifying a location of each of the plurality of anatomical elements based on anatomical context information including location relationships between the plurality of anatomical elements; adjusting the location relationships between the plurality of anatomical elements; and combining the verified and adjusted information of the plurality of anatomical elements with the medical image.

Claims

1. An apparatus for visualizing anatomical elements in a medical image, the apparatus comprising:

an image receiver configured to receive a medical image;

an anatomical element detector configured to detect a plurality of anatomical elements from the medical image;

an analyzer configured to verify a location of each of the plurality of anatomical elements based on anatomical context information comprising location relationships between the plurality of anatomical elements, and adjust the location relationships between the plurality of anatomical elements; and

an image combiner configured to combine the verified and adjusted information of the plurality of anatomical elements with the medical image.

2. The apparatus of claim 1, wherein the medical image is a breast ultrasound image of a human breast captured using ultrasonic waves;

the anatomical element detector is further configured to detect, from the breast ultrasound image, any two or more of skin, fat, glandular tissue, muscle, and bone;

the anatomical context information comprises location relationships between the any two or more of skin, fat, glandular tissue, muscle, and bone; and

the analyzer is further configured to perform the verification and the adjustment based on the anatomical context information comprising the location relationships between the any two or more of skin, fat, glandular tissue, muscle, and bone.

3. The apparatus of claim 1, wherein the anatomical element detector further comprises a plurality of individual detectors each configured to detect a respective one of the plurality of anatomical elements from the medical image.

4. The apparatus of claim 3, wherein each of the plurality of individual detectors is further configured to detect the respective one of the plurality of anatomical elements using any one of a deep learning technique, a sliding window technique, and a superpixel technique.

5. The apparatus of claim 1, wherein the anatomical element detector comprises a single detector configured to detect the plurality of anatomical elements from the medical image simultaneously.

6. The apparatus of claim 5, wherein the single detector is further configured to detect the plurality of anatomical elements from the medical image simultaneously using any one of a deep learning technique, a sliding window technique, and a superpixel technique.

7. The apparatus of claim 5, wherein the single detector is further configured to detect the plurality of anatomical elements from the medical image simultaneously by extracting a plurality of feature maps from the medical image, allocating the plurality of feature maps to corresponding respective anatomical elements, and labeling a location in the medical image of a feature map allocated to a specific anatomical element as the specific anatomical element.

8. The apparatus of claim 1, wherein the analyzer is further configured to adjust a detection result of the plurality of anatomical elements detected from the medical image by the anatomical element detector using a conditional random field technique or a Markov random field technique.

9. The apparatus of claim 1, wherein the anatomical context information further comprises information on a probability distribution of a location at which each of the anatomical elements is located in the medical image;

and

the probability distribution is acquired from pre-established learning data.

10. The apparatus of claim 1, wherein the medical image is one of a plurality of continuous frames or one of a plurality of three-dimensional (3D) images;

and

the anatomical context information further comprises adjacent image information comprising location information of anatomical elements detected from an adjacent frame or an adjacent cross-section.

11. The apparatus of claim 1, wherein the verified and adjusted information of the plurality of anatomical elements to be combined with the medical image comprises area information indicating an area of each of the anatomical elements in the medical image.

12. The apparatus of claim 11, wherein the area information comprises any one or any combination of a contour of an area of each of the plurality of anatomical elements, text representing a name of each of the plurality of anatomical elements, and a color distinguishing an area of each of the plurality of anatomical elements.

13. The apparatus of claim 11, wherein the verified and adjusted information of the plurality anatomical elements to be combined with the medical image comprises confidence level information indicating a confidence level that each of the plurality of anatomical elements is actually an anatomical element.

14. The apparatus of claim 13, wherein the confidence level information comprises any one or any combination of text displayed within an area of each of the plurality of anatomical elements;

a color of a contour of an area of each of the plurality of anatomical elements; and

a transparency of a contour of an area of each of the plurality of anatomical elements.

15. The apparatus of claim 1, where the verified and adjusted information of the plurality of anatomical elements to be combined with the medical image is selected by a user from the information of the plurality of anatomical elements.

16. The apparatus of claim 1, wherein the verified and adjusted information of the plurality of anatomical elements to be combined with the medical image comprises any one or any combination of the following:

area information about an area of each of the plurality of anatomical elements in the medical image;

confidence level information indicating a confidence level that each of the plurality of anatomical elements is actually an anatomical element;

a color or a transparency of a contour of an area of each of the plurality of anatomical elements;

a color or a transparency of text representing a name of each of the plurality of anatomical elements; and

a transparency of a color layer distinguishing an area of each of the plurality of anatomical elements.

17. The apparatus of claim 1, further comprising a lesion verifier configured to verify whether a region of interest (ROI) detected from the medical image is a lesion based on a lesion detection probability of an anatomical element in which the ROI is located.

18. A method of visualizing anatomical elements in a medical image, the method comprising:

receiving a medical image;

detecting a plurality of anatomical elements from the medical image;

verifying a location of each of the plurality of anatomical elements based on anatomical context information comprising location relationships between the plurality of anatomical elements;

adjusting the location relationships between the plurality of anatomical elements; and

combining the verified and adjusted information of the plurality of anatomical elements with the medical image.

19. The apparatus of claim 18, wherein the medical image is a breast ultrasound image of a human breast captured using ultrasonic waves;

the detecting of the plurality of anatomical elements comprises detecting, from the breast ultrasound image, any two or more of skin, fat, glandular tissue, muscle, and bone;

the anatomical context information comprises location relationships between the any two or more of skin, fat, glandular tissue, muscle, and bone; and

the verifying and the adjusting comprise performing the verifying and the adjusting based on the anatomical context information comprising the location relationships between the any two or more of skin, fat, glandular tissue, muscle, and bone.

20. The apparatus of claim 18, wherein the detecting of the plurality of anatomical elements comprises individually detecting respective ones of the plurality of anatomical elements from the medical image.

21. The method of claim 20, wherein the individually detecting of the respective ones of the plurality of anatomical elements comprises individually detecting the respective ones of the plurality of anatomical elements from the medical image using any one of a deep learning technique, a sliding window technique, and a superpixel technique.

22. The method of claim 18, wherein the detecting of the plurality of anatomical elements comprises simultaneously detecting the plurality of anatomical elements from the medical image.

23. The method of claim 22, wherein the simultaneously detecting of the plurality of anatomical elements comprises simultaneously detecting the plurality of anatomical elements using any one of a deep learning technique, a sliding window technique, and a superpixel technique.

24. The method of claim 22, wherein the simultaneously detecting of the plurality of anatomical elements comprises:

extracting a plurality of feature maps from the medical image;

allocating the plurality of feature maps to corresponding respective anatomical elements; and

labeling a location in the medical image of a feature map allocated to a specific anatomical element as the specific anatomical element.

25. The method of claim 18, wherein the adjusting comprises adjusting a detection result of the plurality of anatomical elements detected from the medical image using a conditional random field technique or a Markov random field technique.

26. The method of claim 18, wherein the anatomical context information comprises information on a probability distribution of a location at which each of the plurality of anatomical elements is located in the medical image;

and

the probability distribution is acquired from pre-established learning data.

27. The method of claim 18, wherein the medical image is one of a plurality of continuous frames or one of a plurality of three-dimensional (3D) images;

and

the anatomical context information further comprises adjacent image information comprising location information of anatomical elements detected from an adjacent frame or an adjacent cross-section.

28. The method of claim 18, wherein the verified and adjusted information of the plurality of anatomical elements to be combined with the medical image comprises area information indicating an area of each of the plurality of anatomical elements in the medical image.

29. The method of claim 28, wherein the area information comprises any one or any combination of a contour of an area of each of the plurality of anatomical elements, text representing a name of each of the plurality of anatomical elements, and a color of an area distinguishing each of the plurality of anatomical elements.

30. The method of claim 28, wherein the verified and adjusted information of the plurality of anatomical elements to be combined with the medical image further comprises confidence level information indicating a confidence level that each of the plurality of anatomical elements is actually an anatomical element.

31. The method of claim 30, wherein the confidence level information comprises any one or any combination of text within an area of each of the plurality of anatomical elements, a color of a contour of an area of each of the plurality of anatomical elements, and a transparency of a contour of an area of each of the plurality of anatomical elements.

32. The method of claim 18, further comprising selecting the verified and adjusted information of the plurality of anatomical elements to be combined with the medical image from the information of the plurality of anatomical elements in accordance with a user'"'"'s instruction.

33. The method of claim 18, wherein the verified and adjusted information of the anatomical elements to be combined with the medical image comprises any one or any combination of the following:

area information about an area of each of the plurality of anatomical elements in the medical image;

confidence level information indicating a confidence level that each of the plurality of anatomical elements is actually an anatomical element;

a color or a transparency of an area of each of the plurality of anatomical elements;

a color or a transparency of text representing a name of each of the plurality of anatomical elements; and

a transparency of a color layer distinguishing an area of each of the plurality of anatomical elements.

34. The method of claim 18, further comprising verifying whether a region of interest (ROI) detected from the medical image is a lesion based on a lesion detection probability of an anatomical element in which the detected ROI is located.

35. A method of visualizing anatomical elements in a medical image, the method comprising:

detecting a plurality of anatomical elements from a medical image;

correcting boundaries of the detected anatomical elements to match actual boundaries of the anatomical elements in the medical image based on anatomical context information comprising location relationships between the plurality of anatomical elements; and

combining anatomical element information of the detected anatomical elements having the corrected boundaries with the medical image.

36. The method of claim 35, further comprising ignoring any detected anatomical element that does not correspond to an actual anatomical element in the medical image based on the anatomical context information.

37. The method of claim 35, further comprising selecting the anatomical element information to be combined with the medical image in accordance with a user'"'"'s instruction.