METHOD AND APPARATUS FOR PROCESSING MEDICAL IMAGE, AND ROBOTIC SURGERY SYSTEM USING IMAGE GUIDANCE

• US 20130035583A1
• Filed: 08/01/2012
• Published: 02/07/2013
• Est. Priority Date: 08/02/2011
• Status: Active Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: PARK, Dong-ryeol, KIM, Yeon-ho
• Application Number:
  US13/564,051
• Publication Number:
  US 20130035583A1
• Time in Patent Office: Days
• US Class Current: 600/411
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/368 changing the image on a display according to the operator's position
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/50 specially adapted for specific body parts; specially adapted for specific clinical applications
• CPC Class Codes: A61B 6/506 for diagnosis of nerves
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

A synthesis image in which medical images captured using different medical image capturing apparatuses are registered is generated by mapping the medical images to each other. The synthesis image may be used for image guidance while a diagnosis or a robotic surgery of a patient is being performed.

Claims

1. A method of processing a medical image, the method comprising:

acquiring medical images captured using a plurality of different multi-modal medical image capturing apparatuses with respect to a predetermined organ;

extracting surface information of the predetermined organ, which is included in each of the medical images, from each of the medical images;

mapping each of the medical images using the extracted surface information; and

generating a synthesis image in which the medical images have been registered, based on the mapping result.

2. The method of claim 1, wherein the extracting of the surface information comprises extracting information indicating at least one of a position and a shape of the surface of the predetermined organ from each of the medical images.

3. The method of claim 1, wherein the mapping of each of the medical images comprises mapping each of the medical images by matching positions of the medical image capturing apparatuses with each other using the extracted surface information.

4. The method of claim 1, further comprising detecting positions of the medical image capturing apparatuses,

wherein the mapping of the medical images comprises:

comparing the extracted surface information with each other; and

matching the positions of the medical image capturing apparatuses with each other based on the comparison result, and

the mapping of the medical images comprises mapping the medical image capturing apparatuses based on the matching result.

5. The method of claim 4, wherein the comparing of the extracted surface information comprises comparing the extracted surface information with each other with respect to a surface of a same portion of the predetermined organ.

6. The method of claim 1, wherein the plurality of multi-modal medical image capturing apparatuses comprises an endoscope apparatus and a non-endoscopic apparatus comprising at least one of an ultrasound apparatus, a computed tomography (CT) apparatus, a magnetic resonance imaging (MRI) apparatus, and a positron emission tomography (PET) apparatus.

7. The method of claim 6, wherein the generated synthesis image is an image in which an image of tissues outside the predetermined organ and surroundings of the tissues outside the predetermined organ included in an image captured by the endoscope apparatus and an image of tissues inside and outside the predetermined organ and surroundings of the tissues inside and outside the predetermined organ included in an image captured by the non-endoscopic apparatus are three-dimensionally represented at the same time.

8. The method of claim 6, wherein the extracting of the surface information comprises:

extracting first surface information indicating at least one of a position and a shape of the surface of the predetermined organ from an endoscopic image captured by the endoscope apparatus; and

extracting second surface information indicating at least one of a position and a shape of the surface of the predetermined organ from a non-endoscopic image captured by the non-endoscopic apparatus.

9. The method of claim 8, wherein the extracting of the first surface information comprises:

acquiring distance information between external tissues of the predetermined organ and its surroundings and the endoscope apparatus;

generating a three-dimensional first surface model corresponding to the endoscopic image using the acquired distance information; and

extracting the first surface information from the generated three-dimensional first surface model.

10. The method of claim 8, wherein the extracting of the second surface information comprises:

acquiring information regarding a boundary indicating the surface of the predetermined organ from the image captured by the non-endoscopic apparatus;

generating a three-dimensional second surface model corresponding to the surface of the predetermined organ using the acquired boundary information; and

extracting the second surface information from the generated three-dimensional second surface model.

11. The method of claim 10, wherein the acquiring of the information regarding a boundary comprises acquiring the boundary by applying at least one of line detection and edge detection to the image captured by the non-endoscopic apparatus.

12. The method of claim 1, wherein the predetermined organ corresponds to a part to be operated on by a surgery robot or an organ around or near the part to be operated on.

13. A computer-readable recording medium storing a computer-readable program for executing the method of claim 1.

14. An apparatus for processing a medical image, the apparatus comprising:

an image acquisition unit for acquiring medical images captured using a plurality of different multi-modal medical image capturing apparatuses with respect to a predetermined organ;

a surface information extractor for extracting surface information of the predetermined organ, which is included in each of the medical images, from each of the medical images;

an image mapping unit for mapping each of the medical images using the extracted surface information; and

a synthesis image generator for generating a synthesis image in which the medical images have been registered, based on the mapping result.

15. The apparatus of claim 14, wherein the surface information extractor extracts information indicating at least one of a position and a shape of the surface of the predetermined organ as the surface information from each of the medical images.

16. The apparatus of claim 14, wherein the image mapping unit maps each of the medical images by matching positions of the medical image capturing apparatuses with each other using the extracted surface information.

17. The apparatus of claim 14, further comprising a detector for detecting positions of the medical image capturing apparatuses,

wherein the image mapping unit comprises:

a comparator for comparing the extracted surface information with each other; and

a position matching unit for matching the positions of the medical image capturing apparatuses with each other based on the comparison result.

18. The apparatus of claim 14, wherein the plurality of multi-modal medical image capturing apparatuses comprise an endoscope apparatus and a non-endoscopic apparatus comprising at least one of an ultrasound apparatus, a computed tomography (CT) apparatus, a magnetic resonance imaging (MRI) apparatus, and a positron emission tomography (PET) apparatus.

19. The apparatus of claim 18, wherein the generated synthesis image is an image in which an image of tissues outside the predetermined organ and surroundings of the tissues outside the predetermined organ included in an image captured by the endoscope apparatus and an image of tissues inside and outside the predetermined organ and surroundings of the tissues inside and outside the predetermined organ included in an image captured by the non-endoscopic apparatus are three-dimensionally represented at the same time.

20. The apparatus of claim 18, wherein the surface information extractor comprises:

a first extractor for extracting first surface information indicating at least one of a position and a shape of the surface of the predetermined organ from an endoscopic image captured by the endoscope apparatus; and

a second extractor for extracting second surface information indicating at least one of a position and a shape of the surface of the predetermined organ from a non-endoscopic image captured by the non-endoscopic apparatus.

21. The apparatus of claim 18, wherein the endoscope apparatus is a laparoscope apparatus, and when the non-endoscopic apparatus comprises the ultrasound apparatus, the ultrasound apparatus is a trans-rectal ultrasound (TRUS) apparatus.

22. The apparatus of claim 14, wherein the predetermined organ corresponds to a part to be operated on by a surgery robot or an organ around the part to be operated on.

23. A robotic surgery system for performing robotic surgery with a surgery robot using guiding images of a part to be operated on, the robotic surgery system comprising:

an endoscope apparatus for capturing medical images of a predetermined organ in a body to be examined;

a non-endoscopic apparatus including at least one of an ultrasound apparatus, a computed tomography (CT) apparatus, a magnetic resonance imaging (MRI) apparatus, and a positron emission tomography (PET) apparatus for capturing medical images of the predetermined organ;

a medical image processing apparatus for acquiring the medical images captured using the plurality of multi-modal medical image capturing apparatuses, extracting surface information of the predetermined organ, which is included in each of the medical images, from each of the medical images, mapping each of the medical images using the extracted surface information, and generating a synthesis image in which the medical images have been registered, based on the mapping result;

a display apparatus for displaying the generated synthesis image; and

the surgery robot for performing a robotic surgery.

24. The robotic surgery system of claim 23, wherein the medical image processing apparatus extracts information indicating at least one of a position and a shape of the surface of the predetermined organ as the surface information from each of the medical images.

25. The robotic surgery system of claim 23, wherein the medical image processing apparatus maps the medical images by matching positions of the medical image capturing apparatuses with each other using the extracted surface information.

26. The robotic surgery system of claim 23, wherein the predetermined organ corresponds to a part to be operated on by the surgery robot or an organ around or near the part to be operated on.