Endoscope system with image pasting on planar model

• US 10,413,157 B2
• Filed: 03/25/2016
• Issued: 09/17/2019
• Est. Priority Date: 09/27/2013
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Minamizato, Miho, Hasegawa, Jun
• Primary Examiner(s): Neal, Timothy J
• Assistant Examiner(s): Chou, William B
• Application Number:
  US15/080,768
• Publication Number:
  US 20160278612A1
• Time in Patent Office: 1,271 Days
• Field of Search: 600103, 600109, 600113, 600117, 600118, 600160, 600173, 600176, 600182
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/303 for the vagina, i.e. vaginoscopes
• CPC Class Codes: A61B 1/307 for the urinary organs, e.g. urethroscopes, cystoscopes
• CPC Class Codes: A61B 5/062 using magnetic field

Abstract

An endoscope system includes: an insertion portion to be inserted into a predetermined luminal organ in a subject; an image pickup section provided in a vicinity of a distal end of the insertion portion, the image pickup section picking up an image of a predetermined range; an image pickup information acquiring section acquiring position information and gaze information about the image pickup section; a model image generating section generating a planar model image of the predetermined luminal organ; an image pasting processing section generating a pasted image obtained by pasting a subject image of an inside of the predetermined luminal organ onto the planar model image based on the position information and gaze information about the image pickup section; and an image pasting/presentation processing section presenting an image pickup range of the image pickup section on the pasted image based on the position information about the image pickup section.

Claims

1. An endoscope system comprising:

an insertion portion configured to be inserted into a predetermined luminal organ in a subject;

an image sensor provided in a vicinity of a distal end of the insertion portion, wherein the image sensor is configured to pick up an image of an inside of the predetermined luminal organ; and

a processor comprising hardware, wherein the processor is configured to;

acquire position information and gaze information about the image sensor;

generate a planar model image by planarly developing an inner surface of a three-dimensional model image corresponding to the predetermined luminal organ;

judge whether or not an endoscopic image obtained by picking up the image of the inside of the predetermined luminal organ by the image sensor conforms to a pasting condition;

paste, as a pasted image, the endoscopic image judged to conform to the pasting condition, on the planar model image, based on the position information and gaze information about the image sensor; and

present an image pickup range of the image sensor on the planar model image on which the pasted image is pasted based on the position information about the image sensor.

2. The endoscope system according to claim 1,

wherein the processor is configured to change an angle for presenting the endoscopic image pasted, as the pasted image, on the planar model image based on the position information and the gaze information about the image sensor.

3. The endoscope system according to claim 1,

wherein the processor is configured to change a method for presentation of the pasted image on the planar model image according to a kind of the image sensor.

4. The endoscope system according to claim 1,

wherein the processor is configured to perform presentation of the pasted image so that a position of the pasted image in the planar model image is located at a center of the planar model image, based on the position information about the image sensor.

5. The endoscope system according to claim 1,

wherein the insertion portion comprises a bendable bending portion provided at the distal end of the insertion portion,

wherein the image sensor is provided in a vicinity of a distal end of the bendable bending portion, and

wherein the processor is configured to:

cause the endoscopic image to be displayed on a display; and

perform adjustment so that orientations of the endoscopic image displayed on the display and of the pasted image pasted on the planar model image are caused to correspond to each other, according to a bending state of the image sensor accompanying bending of the bendable bending portion.

6. The endoscope system according to claim 1,

wherein the processor is configured to:

generate the three-dimensional model image corresponding to the predetermined luminal organ in addition to the planar model image;

detect information about an insertion shape of the insertion portion being inserted into the predetermined luminal organ; and

present the information about the insertion shape in association with the three-dimensional model image corresponding to the predetermined luminal organ.

7. The endoscope system according to claim 1, wherein, when pasting a next picked-up image after pasting the image picked up by the image pickup section at a corresponding position on the planar model image as a picked-up image and as a first pasted image so that the picked-up image substantially corresponds to an area corresponding to the picked-up image, the image pasting/presentation processing section changes the first pasted image to a reduced pasted image.

8. The endoscope system according to claim 1, further comprising:

an image display apparatus displaying the picked-up image picked up by the image pickup section and the planar model image in a first display area and a second display area different from each other, respectively, on a same display screen; and

a presentation angle adjusting section adjusting an orientation or an angle of the surface model image so that, relative to upward, downward, right and left directions in a case of the picked-up image being displayed in the first display area, upward, downward, right and left directions of the image pasted on the planar model image displayed in the second display area by the image pasting/presentation processing section correspond, respectively, on the display screen.

9. The endoscope system according to claim 1,

wherein the processor is configured to:

judge whether or not a next endoscopic image, obtained by picking up a next image of the inside of the predetermined luminal organ by the image sensor, conforms to the pasting condition;

paste, as a next pasted image, the next endoscopic image judged to conform to the pasting condition, on the planar model image; and

at a time of pasting the next pasted image on the planar model image, reduce a side of the pasted image pasted on the planar model image.

10. The endoscope system according to claim 1,

wherein the processor is configured to:

control a display to display the endoscopic image in a first display area and the planar model image having the pasted image pasted thereon in a second display area; and

adjust an orientation or an angle of the planar model image so that, relative to upward, downward, right and left directions in a case of the endoscopic image being displayed in the first display area, and upward, downward, right and left directions of the pasted image pasted on the planar model image displayed in the second display area correspond, respectively, on the display.

11. The endoscope system according to claim 1,

wherein the processor is configured to:

calculate a distance from the image sensor to an inner wall of the predetermined luminal organ based on the position information and the gaze information about the image sensor;

calculate an angle formed by a direction of a normal line of the inner wall and a gaze direction of the image sensor based on the position information and the gaze information about the image sensor; and

judge that the endoscopic image conforms to the pasting condition when the distance from the image sensor to the inner wall of the predetermined luminal organ is within a predetermined range, and the angle formed by the direction of the normal line of the inner wall and the gaze direction of the image sensor is small than a predetermined threshold value.