ENDOSCOPE OBSERVATION ASSISTANCE SYSTEM, METHOD, APPARATUS AND PROGRAM

• US 20130018255A1
• Filed: 03/30/2011
• Published: 01/17/2013
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kitamura, Yoshiro, Nakamura, Keigo
• Granted Patent: US 9,220,468 B2
• Time in Patent Office: Days
• US Class Current: 600/424
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2210/41 Medical

Abstract

A virtual endoscope image generation unit receives, as an input, a three-dimensional medical image formed by a three-dimensional medical image formation unit, and generates a virtual endoscope image representing the body cavity viewed from a position in the three-dimensional medical image corresponding to the position of a treatment tool detected in real time by a treatment tool position detection unit. The generated virtual endoscope image is displayed on a WS display.

Claims

1-19. -19. (canceled)

20. An endoscope observation assistance system comprising:

a three-dimensional medical image formation unit that forms a three-dimensional medical image representing the body cavity of a subject to be examined;

a treatment tool position detection unit that detects, in real time, the position of a treatment tool inserted into the body cavity;

a position-of-interest specifying unit that specifies, as a position of interest, the position of a structure of interest in the body cavity in the three-dimensional medical image;

a virtual endoscope image generation unit that receives the three-dimensional medical image as an input, and generates a virtual endoscope image representing the body cavity viewed from a corresponding treatment tool position, which is a position in the three-dimensional medical image corresponding to the detected position of the treatment tool; and

a display unit that displays the virtual endoscope image,

wherein the virtual endoscope image generation unit generates the virtual endoscope image, the field of view of which includes the specified position of interest.

21. An endoscope observation assistance system, as defined in claim 20, wherein the virtual endoscope image generation unit generates the virtual endoscope image by determining the angle of view of the virtual endoscope image in such a manner that the specified position of interest is included in a field of view of the virtual endoscope image.

22. An endoscope observation assistance system, as defined in claim 20, wherein the treatment tool position detection unit detects a direction in which the treatment tool moves straight forward at the same time as detection of the position of the treatment tool, and

wherein the virtual endoscope image generation unit generates the virtual endoscope image in which image information on a treatment tool visual line from the corresponding treatment tool position is projected, and the treatment tool visual line having a direction in the three-dimensional medical image corresponding to the direction in which the treatment tool moves straight forward.

23. An endoscope observation assistance system, as defined in claim 22, wherein the virtual endoscope image generation unit generates the virtual endoscope image that identifiably represents a position in the virtual endoscope image at which the image information on the treatment tool visual line has been projected.

24. An endoscope observation assistance system, as defined in claim 22, wherein the virtual endoscope image generation unit generates the virtual endoscope image in which the treatment tool visual line is a center visual line, which is a visual line at the center of the field of view of the virtual endoscope image.

25. An endoscope observation assistance system, as defined in claim 20, wherein the virtual endoscope image generation unit generates the virtual endoscope image in which a visual line from the corresponding treatment tool position toward the specified position of interest is a center visual line of the virtual endoscope image.

26. An endoscope observation assistance system, as defined in claim 20, wherein the virtual endoscope image generation unit generates the virtual endoscope image that identifiably represents at least the position of interest in the structure of interest.

27. An endoscope observation assistance system, as defined in claim 20, the system further comprising:

a warning unit that presents a warning when the corresponding treatment tool position and the structure of interest are close to each other to such an extent to satisfy a predetermined criterion.

28. An endoscope observation assistance system, as defined in claim 20, the system further comprising:

a second-structure-of-interest detection unit that detects, in the three-dimensional medical image, a second structure of interest in the body cavity,

wherein the virtual endoscope image generation unit generates the virtual endoscope image that identifiably represents the second structure of interest.

29. An endoscope observation assistance system, as defined in claim 20, wherein the structure of interest is a region on which an endoscopic surgery is to be performed.

30. An endoscope observation assistance system, as defined in

claim 20, wherein the structure of interest is an anatomical structure that requires caution in an endoscopic surgery.

31. An endoscope observation assistance system, as defined in claim 28, wherein one of the structure of interest and the second structure of interest is a region on which an endoscopic surgery is to be performed, and the other one of the structure of interest and the second structure of interest is an anatomical structure that requires caution in the endoscopic surgery.

32. An endoscope observation assistance system, as defined in claim 20, wherein the virtual endoscope image generation unit determines pixel values of the virtual endoscope image based on a distance from the corresponding treatment tool position to a surface of a structure in the body cavity.

33. An endoscope observation assistance system, as defined in claim 20, wherein the virtual endoscope image generation unit determines pixel values of the virtual endoscope image by using a color template that is defined so that an external view of each part in the body cavity represented in the virtual endoscope image is substantially the same as an external view of each part in the body cavity represented in a real endoscope image obtained by imaging by an endoscope.

34. An endoscope observation assistance system, as defined in claim 20, the system further comprising:

an endoscope position detection unit that detects the position of the endoscope inserted into the body cavity,

wherein the virtual endoscope image generation unit generates the virtual endoscope image, the field of view of which includes a corresponding endoscope position, which is a position in the three-dimensional medical image corresponding to the detected position of the endoscope, and the virtual endoscope image identifiably representing the corresponding endoscope position.

35. An endoscope observation assistance system, as defined in claim 20, the system further comprising:

a real endoscope image formation unit that forms a real endoscope image representing the body cavity by imaging in real time by an endoscope,

wherein the display unit further displays the real endoscope image formed substantially at the same timing as detection of the position of the treatment tool used at the time of generation of the virtual endoscope image.

36. An endoscope observation assistance method comprising the steps of:

forming a three-dimensional medical image representing the body cavity of a subject to be examined;

detecting, in real time, the position of a treatment tool inserted into the body cavity;

specifying, as a position of interest, the position of a structure of interest in the body cavity in the three-dimensional medical image;

receiving the three-dimensional medical image as an input, and generating a virtual endoscope image representing the body cavity viewed from a corresponding treatment tool position, which is a position in the three-dimensional medical image corresponding to the detected position of the treatment tool; and

displaying the virtual endoscope image,

wherein the step of generating the virtual endoscope image generates the virtual endoscope image, the field of view of which includes the specified position of interest.

37. An endoscope observation assistance apparatus comprising:

a three-dimensional medical image obtainment unit that obtains a three-dimensional medical image representing the body cavity of a subject to be examined;

a treatment tool position obtainment unit that obtains the position of a treatment tool inserted into the body cavity, and the position having been detected in real time by a treatment tool position detection unit;

a position-of-interest specifying unit that specifies, as a position of interest, the position of a structure of interest in the body cavity in the three-dimensional medical image;

a virtual endoscope image generation unit that receives the three-dimensional medical image as an input, and generates a virtual endoscope image representing the body cavity viewed from a corresponding treatment tool position, which is a position in the three-dimensional medical image corresponding to the obtained position of the treatment tool; and

a display control unit that makes a display unit display the virtual endoscope image,

wherein the virtual endoscope image generation unit generates the virtual endoscope image, the field of view of which includes the specified position of interest.

38. A non-transitory computer-readable storage medium storing therein an endoscope observation assistance program for causing a computer to execute the steps of:

obtaining a three-dimensional medical image representing the body cavity of a subject to be examined;

obtaining the position of a treatment tool inserted into the body cavity, and the position having been detected in real time by a treatment tool position detection unit;

specifying, as a position of interest, the position of a structure of interest in the body cavity in the three-dimensional medical image;

receiving the three-dimensional medical image as an input, and generating a virtual endoscope image representing the body cavity viewed from a corresponding treatment tool position, which is a position in the three-dimensional medical image corresponding to the obtained position of the treatment tool; and

making a display unit display the virtual endoscope image, wherein the step of generating the virtual endoscope image generates the virtual endoscope image, the field of view of which includes the specified position of interest.