Endoscopic observation supporting system, method, device and program

• US 9,179,822 B2
• Filed: 03/16/2011
• Issued: 11/10/2015
• Est. Priority Date: 03/17/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kitamura, Yoshiro, Nakamura, Keigo
• Primary Examiner(s): YENKE, BRIAN P
• Application Number:
  US13/582,603
• Publication Number:
  US 20120327186A1
• Time in Patent Office: 1,700 Days
• Field of Search: 348/45, 348/166, 382/128, 434/262, 600/111, 600/116
• US Class Current: 1/1
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound

Abstract

A virtual endoscopic image generating unit for generating, from a 3D medical image representing an interior of a body cavity of a subject formed by a 3D medical image forming unit and inputted thereto, a virtual endoscopic image, in which a position of a structure of interest identified by a position of interest identifying unit is a view point of the virtual endoscopic image, a real-time position of at least one of an endoscope and a surgical tool detected by an endoscope position detecting unit or a surgical tool position detecting unit is contained in a field of view of the virtual endoscopic image, and the position of at least one of the endoscope and the surgical tool is shown in an identifiable manner in the virtual endoscopic image, is provided. A display control unit causes a WS display to display the generated virtual endoscopic image.

Claims

1. An endoscopic observation support system comprising:

a 3D medical image forming unit configured to form a 3D medical image representing an interior of a body cavity of a subject;

a position of interest identifying unit configured to identify a position of a structure of interest in the body cavity in the 3D medical image, the structure of interest being a site of surgical interest or an anatomical structure that requires attention during endoscopic surgery;

a position detecting unit configured to detect a real-time position of at least one of an endoscope and a surgical tool inserted in the body cavity;

a virtual endoscopic image generating unit configured to generate, from the 3D medical image inputted thereto, a virtual endoscopic image representing the interior of the body cavity viewed from a view point, based on the identified position of the structure of interest and the detected position of at least one of the endoscope and the surgical tool in the 3D medical image, wherein the view point is the position of the structure of interest, the position of at least one of the endoscope and the surgical tool is contained in a field of view of the virtual endoscopic image, and the position of at least one of the endoscope and the surgical tool is shown in an identifiable manner in the virtual endoscopic image; and

a display control unit configured to cause a display unit to display the virtual endoscopic image.

2. The endoscopic observation support system as claimed in claim 1, further comprising

a real endoscopic image forming unit configured to form a real endoscopic image representing the interior of body cavity by real-time imaging with the endoscope,

wherein the display control unit further causes the real endoscopic image which is formed almost at the same time when the position of at least one of the endoscope and the surgical tool used to generate the virtual endoscopic image is detected to be displayed.

3. The endoscopic observation support system as claimed in claim 1, wherein the virtual endoscopic image generating unit determines pixel values of the virtual endoscopic image depending on a distance from the structure of interest to a surface of a structure in the body cavity.

4. The endoscopic observation support system as claimed in claim 1, further comprising a warning unit configured to show a warning when an approach of at least one of the endoscope and the surgical tool to the structure of interest satisfies a predetermined criterion.

5. The endoscopic observation support system as claimed in claim 1, wherein the virtual endoscopic image generating unit determines pixel values of the virtual endoscopic image using a color template, wherein the color template is defined to provide the virtual endoscopic image showing sites in the body cavity in almost the same appearance as those shown in a real endoscopic image obtained by imaging with the endoscope.

6. The endoscopic observation support system as claimed in claim 1, further comprising

a second structure of interest detecting unit configured to detect a second structure of interest in the body cavity in the 3D medical image,

wherein the virtual endoscopic image generating unit generates the virtual endoscopic image in which the second structure of interest is shown in a visually recognizable manner.

7. The endoscopic observation support system as claimed in claim 6, wherein the structure of interest is a site of surgical interest during endoscopic surgery using the endoscope and the second structure of interest is an anatomical structure that requires attention during the endoscopic surgery.

8. The endoscopic observation support system as claimed in claim 1, wherein the position of the endoscope is contained in a field of view of the virtual endoscopic image, and the position of the endoscope is shown in an identifiable manner in the virtual endoscopic image.

9. An endoscopic observation support method comprising the steps of:

forming a 3D medical image representing an interior of a body cavity of a subject before or during observation of the interior of the body cavity with an endoscope inserted in the body cavity;

identifying a position of a structure of interest in the body cavity in the 3D medical image, the structure of interest being a site of surgical interest or an anatomical structure that requires attention during endoscopic surgery;

detecting a real-time position of at least one of the endoscope and a surgical tool inserted in the body cavity;

generating, from the 3D medical image inputted, a virtual endoscopic image representing the interior of the body cavity viewed from a view point, based on the identified position of the structure of interest and the detected position of at least one of the endoscope and the surgical tool in the 3D medical image, wherein the view point is the position of the structure of interest, the position of at least one of the endoscope and the surgical tool is contained in a field of view of the virtual endoscopic image, and the position of at least one of the endoscope and the surgical tool is shown in an identifiable manner in the virtual endoscopic image; and

displaying the virtual endoscopic image.

10. The endoscopic observation support method as claimed in claim 9, wherein the position of the endoscope is contained in a field of view of the virtual endoscopic image, and the position of the endoscope is shown in an identifiable manner in the virtual endoscopic image.

11. An endoscopic observation support device comprising:

a 3D medical image obtaining unit configured to obtain a 3D medical image representing an interior of a body cavity of a subject;

a position of interest identifying unit configured to identify a position of a structure of interest in the body cavity in the 3D medical image, the structure of interest being a site of surgical interest or an anatomical structure that requires attention during endoscopic surgery;

a position obtaining unit configured to obtain a real-time position of at least one of an endoscope and a surgical tool inserted in the body cavity detected by a position detecting unit;

a virtual endoscopic image generating unit configured to generate, from the 3D medical image inputted thereto, a virtual endoscopic image representing the interior of the body cavity viewed from a view point, based on the identified position of the structure of interest and the detected position of at least one of the endoscope and the surgical tool in the 3D medical image, wherein the view point is the position of the structure of interest, the position of at least one of the endoscope and the surgical tool is contained in a field of view of the virtual endoscopic image, and the position of at least one of the endoscope and the surgical tool is shown in an identifiable manner in the virtual endoscopic image; and

a display control unit configured to cause a display unit to display the virtual endoscopic image.

12. The endoscopic observation support device as claimed in claim 11, wherein the position of the endoscope is contained in a field of view of the virtual endoscopic image, and the position of the endoscope is shown in an identifiable manner in the virtual endoscopic image.

13. A non-transitory computer readable medium containing an endoscopic observation support program for causing a computer to carry out the steps of:

obtaining a 3D medical image representing an interior of a body cavity of a subject;

identifying a position of a structure of interest in the body cavity in the 3D medical image, the structure of interest being a site of surgical interest or an anatomical structure that requires attention during endoscopic surgery;

obtaining a real-time position of at least one of an endoscope and a surgical tool inserted in the body cavity detected by a position detecting unit;

generating, from the 3D medical image inputted, a virtual endoscopic image representing the interior of the body cavity viewed from a view point, based on the identified position of the structure of interest and the detected position of at least one of the endoscope and the surgical tool in the 3D medical image, wherein the view point is the position of the structure of interest, the position of at least one of the endoscope and the surgical tool is contained in a field of view of the virtual endoscopic image, and the position of at least one of the endoscope and the surgical tool is shown in an identifiable manner in the virtual endoscopic image; and

causing a display unit to display the virtual endoscopic image.

14. The non-transitory computer readable medium as claimed in claim 13, wherein the position of the endoscope is contained in a field of view of the virtual endoscopic image, and the position of the endoscope is shown in an identifiable manner in the virtual endoscopic image.