Endoscopic observation support system

• US 9,375,133 B2
• Filed: 03/30/2011
• Issued: 06/28/2016
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kitamura, Yoshiro, Nakamura, Keigo
• Primary Examiner(s): Leubecker, John P
• Assistant Examiner(s): TORRES DIAZ, ARNALDO
• Application Number:
  US13/638,812
• Publication Number:
  US 20130023730A1
• Time in Patent Office: 1,917 Days
• Field of Search: 600/104, 600/111, 600/124, 600424-427, 600/476
• US Class Current: 1/1
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/466 adapted to display 3D data
• CPC Class Codes: A61B 6/486 involving generating temporal series of image data
• CPC Class Codes: A61B 6/5211 involving processing of medical diagnostic data
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A virtual field of view of a virtual endoscope, which is positioned at a position corresponding to a detected position of an endoscope in a 3D medical image, is determined based on a position of a structure of interest, the corresponding position and a posture of the endoscope, and an angle of view of the endoscope, such that the position of the structure of interest is contained within the virtual field of view and the virtual field of view has continuity with the field of view of the endoscope. From the 3D medical image inputted, a virtual endoscopic image having the determined virtual field of view with the view point thereof being the corresponding position of the endoscope is generated. The generated virtual endoscopic image is displayed on a WS display.

Claims

1. An endoscopic observation support system comprising:

a 3D medical image forming unit configured to form a 3D medical image representing an interior of a body cavity of a subject;

a position of interest identifying unit configured to identify a position of a structure of interest in the body cavity in the 3D medical image;

an endoscope position and posture detecting unit configured to detect a real-time position and a real-time posture of an endoscope inserted in the body cavity;

an endoscope angle of view obtaining unit configured to obtain information of an angle of view of the endoscope;

a virtual field of view determining unit configured to determine a virtual field of view of a virtual endoscope positioned at an endoscope-corresponding position, the endoscope-corresponding position being a position in the 3D medical image corresponding to the detected position of the endoscope, based on the identified position of the structure of interest, the detected position and posture of the endoscope in the 3D medical image, and the angle of view of the endoscope, such that the position of the structure of interest is contained within the virtual field of view and the virtual field of view has continuity with an endoscope-corresponding field of view, the endoscope-corresponding field of view being a field of view of the 3D medical image corresponding to a field of view of the endoscope;

a virtual endoscopic image generating unit configured to generate, from the 3D medical image inputted thereto, a virtual endoscopic image having the virtual field of view with a view point thereof being the endoscope-corresponding position;

a display control unit configured to cause a display unit to display the virtual endoscopic image; and

a real endoscopic image forming unit configured to form a real endoscopic image representing the interior of the body cavity by real-time imaging with the endoscope,

wherein the display control unit causes the virtual endoscopic image and the real endoscopic image which is formed almost at the same time when the position and posture of the endoscope used to generate the virtual endoscopic image are detected to be displayed with being superimposed a part of the real endoscopic image on a part of the virtual endoscopic image which is the same area of an organ in the interior of the body cavity that is represented in the virtual endoscopic image as the area of the organ in the interior of the body cavity that is represented in the real endoscopic image.

2. The endoscopic observation support system as claimed in claim 1, wherein the virtual endoscopic image generating unit generates the virtual endoscopic image in which the structure of interest is shown in an identifiable manner.

3. The endoscopic observation support system as claimed in claim 1, wherein the virtual endoscopic image generating unit determines pixel values of the virtual endoscopic image depending on a distance from the position of the endoscope to a surface of a structure in the body cavity.

4. The endoscopic observation support system as claimed in claim 1, further comprising a warning unit configured to show a warning when an approach of the endoscope to the structure of interest satisfies a predetermined criterion.

5. The endoscopic observation support system as claimed in claim 1, wherein the virtual endoscopic image generating unit determines pixel values of the virtual endoscopic image using a color template, wherein the color template is defined to provide the virtual endoscopic image showing sites in the body cavity in almost the same appearance as those shown in the real endoscopic image obtained by imaging with the endoscope.

6. The endoscopic observation support system as claimed in claim 1, further comprising

a second position of interest identifying unit configured to identify a position of a second structure of interest in the body cavity in the 3D medical image,

wherein the virtual endoscopic image generating unit generates the virtual endoscopic image in which the second structure of interest is shown in an identifiable manner.

7. The endoscopic observation support system as claimed in claim 6, wherein the structure of interest is a site of surgical interest during endoscopic surgery using the endoscope and the second structure of interest is an anatomical structure that requires attention during the endoscopic surgery.

8. The endoscopic observation support system as claimed in claim 6, wherein the structure of interest is a site of surgical interest during endoscopic surgery using the endoscope, and the second structure of interest is a surgical tool inserted in the body cavity.

9. The endoscopic observation support system as claimed in claim 1, wherein the structure of interest is a site of surgical interest during endoscopic surgery using the endoscope.

10. The endoscopic observation support system as claimed in claim 1, wherein the structure of interest is an anatomical structure that requires attention during endoscopic surgery using the endoscope.

11. The endoscopic observation support system as claimed in claim 1, wherein the structure of interest is a surgical tool inserted in the body cavity.

12. An endoscopic observation support system comprising:

a 3D medical image forming unit configured to form a 3D medical image representing an interior of a body cavity of a subject;

a position of interest identifying unit configured to identify a position of a structure of interest in the body cavity in the 3D medical image;

an endoscope position and posture detecting unit configured to detect a real-time position and a real-time posture of an endoscope inserted in the body cavity;

an endoscope angle of view obtaining unit configured to obtain information of an angle of view of the endoscope;

a virtual field of view determining unit configured to determine a virtual field of view of a virtual endoscope positioned at an endoscope-corresponding position, the endoscope-corresponding position being a position in the 3D medical image corresponding to the detected position of the endoscope, based on the identified position of the structure of interest, the detected position and posture of the endoscope in the 3D medical image, and the angle of view of the endoscope, such that the position of the structure of interest is contained within the virtual field of view and the virtual field of view has continuity with an endoscope-corresponding field of view, the endoscope-corresponding field of view being a field of view of the 3D medical image corresponding to a field of view of the endoscope;

a virtual endoscopic image generating unit configured to generate, from the 3D medical image inputted thereto, a virtual endoscopic image having the virtual field of view with a view point thereof being the endoscope-corresponding position; and

a display control unit configured to cause a display unit to display the virtual endoscopic image,

wherein the virtual field of view determining unit determines the virtual field of view such that an angle of view of the virtual endoscope is wider than the angle of view of the endoscope.

13. An endoscopic observation support method comprising the steps of:

forming a 3D medical image representing an interior of a body cavity of a subject before or during endoscopic observation of the interior of the body cavity with an endoscope inserted in the body cavity;

identifying a position of a structure of interest in the body cavity in the 3D medical image;

detecting a real-time position and a real-time posture of the endoscope inserted in the body cavity;

obtaining information of an angle of view of the endoscope;

determining a virtual field of view of a virtual endoscope positioned at an endoscope-corresponding position, the endoscope-corresponding position being a position in the 3D medical image corresponding to the detected position of the endoscope, based on the identified position of the structure of interest, the detected position and posture of the endoscope in the 3D medical image, and the angle of view of the endoscope, such that the position of the structure of interest is contained within the virtual field of view and the virtual field of view has continuity with an endoscope-corresponding field of view, the endoscope-corresponding field of view being a field of view of the 3D medical image corresponding to a field of view of the endoscope;

generating, from the 3D medical image inputted, a virtual endoscopic image having the virtual field of view with a view point thereof being the endoscope-corresponding position, the virtual endoscopic image representing the interior of the body cavity viewed from the view point; and

displaying the virtual endoscopic image,

wherein the virtual field of view determining determines the virtual field of view such that an angle of view of the virtual endoscope is wider than the angle of view of the endoscope.

14. An endoscopic observation support device comprising:

a 3D medical image obtaining unit configured to obtain a 3D medical image representing an interior of a body cavity of a subject;

a position of interest identifying unit configured to identify a position of a structure of interest in the body cavity in the 3D medical image;

a position and posture obtaining unit configured to obtain a real-time position and a real-time posture of an endoscope inserted in the body cavity detected by a position and posture detecting unit;

an endoscope angle of view obtaining unit configured to obtain information of an angle of view of the endoscope;

a virtual field of view determining unit configured to determine a virtual field of view of a virtual endoscope positioned at an endoscope-corresponding position, the endoscope-corresponding position being a position in the 3D medical image corresponding to the detected position of the endoscope, based on the identified position of the structure of interest, the obtained position and posture of the endoscope in the 3D medical image, and the angle of view of the endoscope, such that the position of the structure of interest is contained within the virtual field of view and the virtual field of view has continuity with an endoscope-corresponding field of view, the endoscope-corresponding field of view being a field of view of the 3D medical image corresponding to a field of view of the endoscope;

a virtual endoscopic image generating unit configured to generate, from the 3D medical image inputted thereto, a virtual endoscopic image having the virtual field of view with a view point thereof being the endoscope-corresponding position; and

a display control unit configured to cause a display unit to display the virtual endoscopic image, wherein the virtual field of view determining unit determines the virtual field of view such that an angle of view of the virtual endoscope is wider than the angle of view of the endoscope.

15. A non-transitory computer readable medium containing an endoscopic observation support program for causing a computer to carry out the steps of:

obtaining a 3D medical image representing an interior of a body cavity of a subject;

identifying a position of a structure of interest in the body cavity in the 3D medical image;

obtaining a real-time position and a real-time posture of an endoscope inserted in the body cavity detected by a position and posture detecting unit;

obtaining information of an angle of view of the endoscope;

determining a virtual field of view of a virtual endoscope positioned at an endoscope-corresponding position, the endoscope-corresponding position being a position in the 3D medical image corresponding to the detected position of the endoscope, based on the identified position of the structure of interest, the detected position and posture of the endoscope in the 3D medical image, and the angle of view of the endoscope, such that the position of the structure of interest is contained within the virtual field of view and the virtual field of view has continuity with an endoscope-corresponding field of view, the endoscope-corresponding field of view being a field of view of the 3D medical image corresponding to a field of view of the endoscope;

generating, from the 3D medical image inputted, a virtual endoscopic image having the virtual field of view with a view point thereof being the endoscope-corresponding position; and

causing a display unit to display the virtual endoscopic image,

wherein the virtual field of view determining determines the virtual field of view such that an angle of view of the virtual endoscope is wider than the angle of view of the endoscope.