Endoscope system and method for operating endoscope system that display an organ model image to which an endoscopic image is pasted

• US 9,460,536 B2
• Filed: 07/28/2014
• Issued: 10/04/2016
• Est. Priority Date: 03/06/2013
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Hasegawa, Jun, Onishi, Junichi, Akimoto, Syunya
• Primary Examiner(s): Leubecker, John P
• Application Number:
  US14/444,211
• Publication Number:
  US 20150025316A1
• Time in Patent Office: 799 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 1/307 for the urinary organs, e.g. urethroscopes, cystoscopes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion

Abstract

An endoscope system includes an insertion portion, an objective optical window, an image pickup device, a position/direction detection section that acquires position information of the objective optical window, and a memory that records the subject internal image acquired by the image pickup device in association with the position information of the objective optical window. The endoscope system aligns the position information of the objective optical window with a reference position of a predetermined organ in the subject in a coordinate system of a three-dimensional model image based on an amount of change or the like of the subject internal image information in the subject and generates an image with the subject internal image pasted onto the two-dimensional model image of the predetermined organ which is the three-dimensional model image two-dimensionally developed in which the position of the objective optical window is associated with the position in the coordinate system.

Claims

1. An endoscope system comprising:

an insertion portion that is inserted into a subject;

an objective optical window that is provided on a distal end side of the insertion portion and receives light from the subject;

an image pickup section that picks up an image of an inside of the subject from the light entering from the objective optical window;

a position information acquiring section that acquires position information of the objective optical window;

an alignment section that aligns the position of the objective optical window acquired from the position information acquiring section with a reference position of a predetermined organ in the subject in a coordinate system of a three-dimensional model image based on an amount of change of subject internal image information inside the subject, predetermined operation input or the position information with respect to a preset reference plane; and

an image generating section that generates an image with a subject internal image pasted onto a two-dimensional model image of the predetermined organ obtained by two-dimensionally developing the three-dimensional model image in which the position of the objective optical window is associated with a reference position in the coordinate system by the alignment section,

wherein the position information acquiring section acquires direction information in addition to the position information and further comprises a shape estimation section that performs shape estimation of the insertion portion based on the position information and the direction information, and

the image generating section superimposes shape information estimated by the shape estimation section on the three-dimensional model image relating to the predetermined organ.

2. The endoscope system according to claim 1, wherein the alignment section associates a position and direction of the objective optical window acquired by the position information acquiring section with a position and direction of the three-dimensional model image of the predetermined organ in the coordinate system.

3. The endoscope system according to claim 1, wherein an amount of change of the subject internal image information is an amount of change in luminance, color or texture obtained by being extracted from the subject internal image.

4. The endoscope system according to claim 1, wherein the two-dimensional model image of the predetermined organ is an development diagram of the bladder.

5. The endoscope system according to claim 1, wherein the image generating section displays a live subject internal image acquired by the image pickup section together with the two-dimensional model image.

6. The endoscope system according to claim 5, wherein the image generating section displays an insertion shape of the insertion portion comprising the image pickup section that picks up the live subject internal image.

7. An endoscope system comprising:

an insertion portion that is inserted into a subject;

an objective optical window that is provided on a distal end side of the insertion portion and receives light from the subject;

an image pickup section that picks up an image of an inside of the subject from the light entering from the objective optical window;

a position information acquiring section that acquires position information of the objective optical window;

an alignment section that aligns the position of the objective optical window acquired from the position information acquiring section with a reference position of a predetermined organ in the subject in a coordinate system of a three-dimensional model image based on an amount of change of subject internal image information inside the subject, predetermined operation input or the position information with respect to a preset reference plane; and

an image generating section that generates an image with a subject internal image pasted onto a two-dimensional model image of the predetermined organ obtained by two-dimensionally developing the three-dimensional model image in which the position of the objective optical window is associated with a reference position in the coordinate system by the alignment section,

the endoscope system further comprising an illumination section that irradiates the subject with white color light or special light having a predetermined wavelength band in a switchable manner,

wherein the image generating section sets a plurality of the two-dimensional model images as well as pasting the subject internal image onto a plurality of models set based on the type of illuminating light of the illumination section.

8. The endoscope system according to claim 7, wherein the special light is narrow band light.

9. The endoscope system according to claim 7, wherein an amount of change of the subject internal image information is an amount of change in luminance, color or texture obtained by being extracted from the subject internal image.

10. The endoscope system according to claim 7, wherein the two-dimensional model image of the predetermined organ is an development diagram of the bladder.

11. The endoscope system according to claim 7, wherein the image generating section displays a live subject internal image acquired by the image pickup section together with the two-dimensional model image.

12. The endoscope system according to claim 11, wherein the image generating section displays an insertion shape of the insertion portion comprising the image pickup section that picks up the live subject internal image.

13. An endoscope system comprising:

an insertion portion that is inserted into a subject;

an objective optical window that is provided on a distal end side of the insertion portion and receives light from the subject;

an image pickup section that picks up an image of an inside of the subject from the light entering from the objective optical window;

a position information acquiring section that acquires position information of the objective optical window;

an alignment section that aligns the position of the objective optical window acquired from the position information acquiring section with a reference position of a predetermined organ in the subject in a coordinate system of a three-dimensional model image based on an amount of change of subject internal image information inside the subject, predetermined operation input or the position information with respect to a preset reference plane; and

an image generating section that generates an image with a subject internal image pasted onto a two-dimensional model image of the predetermined organ obtained by two-dimensionally developing the three-dimensional model image in which the position of the objective optical window is associated with a reference position in the coordinate system by the alignment section,

wherein the image generating section displays a live subject internal image acquired by the image pickup section together with the two-dimensional model image, and

wherein the image generating section displays an insertion shape of the insertion portion comprising the image pickup section that picks up the live subject internal image.

14. An method for operating an endoscope system comprising, a position information acquiring section that acquires position information of an objective optical window of an insertion portion of an endoscope that receives light from a subject, an image pickup section that picks up an image of an inside of the subject from light inputted from the objective optical window, an alignment section and an image generating section, the method comprising:

the alignment section aligning the position of the objective optical window acquired from the position information acquiring section with a reference position of a predetermined organ in the subject in a coordinate system of a three-dimensional model image based on an amount of change of subject internal image information inside the subject, predetermined operation input or the position information with respect to a preset reference plane; and

the image generating section associating the position of the objective optical window with a reference position in the coordinate system by the alignment section and generating an image with a subject internal image pasted onto a two-dimensional model image of the predetermined organ obtained by two-dimensionally developing the three-dimensional model image,

wherein the endoscope system comprises a shape estimation section,

the position information acquiring section acquires direction information in addition to the position information,

the shape estimation section performs shape estimation of the insertion portion based on the position information and the direction information, and

the image generating section generates an image to which the subject internal image is pasted by superimposing shape information estimated by the shape estimation section on a three-dimensional model image relating to the predetermined organ.

15. An endoscope system comprising:

an insertion portion that is inserted into a subject;

an objective optical window that is provided on a distal end side of the insertion portion and receives light from the subject;

an image pickup section that picks up an image of an inside of the subject from the light entering from the objective optical window;

a position information acquiring section that acquires position information of the objective optical window;

an alignment section that aligns the position of the objective optical window acquired from the position information acquiring section with a reference position of a predetermined organ in the subject in a coordinate system of a three-dimensional model image based on an amount of change of subject internal image information inside the subject, predetermined operation input or the position information with respect to a preset reference plane; and

an image generating section that generates an image with a subject internal image pasted onto a two-dimensional model image of the predetermined organ obtained by two-dimensionally developing the three-dimensional model image in which the position of the objective optical window is associated with a reference position in the coordinate system by the alignment section,

the endoscope system further comprising an illumination section that irradiates the subject with one of white color light or special light, the special light having a predetermined wavelength band in a switchable manner,

wherein the image generating section selectively pastes, on the two-dimensional model image, any one of two types of subject internal images generated by switching of illumination light by the illumination section.