ENDOSCOPE DEVICE

• US 20190324261A1
• Filed: 04/22/2019
• Published: 10/24/2019
• Est. Priority Date: 04/23/2018
• Status: Active Grant

• Current Assignee: Evident Corporation
• Original Assignee: Olympus Corporation
• Inventors: OGAWA, Kiyotomi
• Granted Patent: US 10,921,577 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: G02B 23/2415 Stereoscopic endoscopes
• CPC Class Codes: G02B 23/2461 Illumination
• CPC Class Codes: G02B 23/2469 using optical fibres
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes

Abstract

In an endoscope device, an imaging device sequentially reads pixel signals from at least some of a plurality of pixels row by row during a first period. A control unit causes a light source to generate illumination light during a second period. The second period is at least a part of a period other than the first period. The control unit causes the light source to stop the generation of the illumination light during a third period. The third period is all of a period other than the second period. The control unit causes a switching unit to start switching of an imaging condition during the third period and complete the switching of the imaging condition during the third period.

Claims

1. An endoscope device comprising:

a light source configured to generate illumination light;

an illumination optical system configured to radiate the illumination light to a subject;

an observation optical system configured to form an optical image of the subject;

an imaging device having a plurality of pixels disposed in a matrix and configured to image the subject, sequentially read pixel signals from at least some of the plurality of pixels row by row during a first period, and generate an image data of the subject during each frame period of a plurality of frame periods on the basis of the pixel signals read from at least some of the plurality of pixels, the pixel signals being generated on the basis of the optical image of the subject;

a switching unit configured to perform switching between a plurality of imaging conditions so that the imaging device images the subject; and

a control unit,

wherein the control unit causes the light source to generate the illumination light during a second period, the second period being at least a part of a period other than the first period,

the control unit causes the light source to stop the generation of the illumination light during a third period, the third period is all of a period other than the second period and includes the first period, and the second period and the third period are alternately iterated, and

the control unit causes the switching unit to start switching of the imaging condition during the third period and complete the switching of the imaging condition during the third period.

2. The endoscope device according to claim 1, wherein the imaging device reads the pixel signals in a time period that is less than or equal to half a length of each frame period of the plurality of frame periods during the first period.

3. The endoscope device according to claim 1, further comprising an image processing unit configured to execute image processing on a plurality of images generated during the plurality of frame periods,

wherein the plurality of imaging conditions include a first imaging condition and a second imaging condition,

the first imaging condition and the second imaging condition are different from each other,

the imaging device generates a first image of the subject by imaging the subject under the first imaging condition,

the imaging device generates a second image of the subject by imaging the subject under the second imaging condition, and

the image processing unit processes the first image and the second image.

4. The endoscope device according to claim 3,

wherein the imaging device generates a plurality of at least one of first images and second images,

when the imaging device generates the plurality of first images, the image processing unit calculates a value indicating whether or not the plurality of first images are suitable for image processing, and

when the imaging device generates the plurality of second images, the image processing unit calculates a value indicating whether or not the plurality of second images are suitable for image processing.

5. The endoscope device according to claim 3,

wherein the imaging device generates a plurality of first images by imaging the subject under the first imaging condition,

the imaging device generates a plurality of second images by imaging the subject under the second imaging condition,

the image processing unit generates a third image by executing a noise reduction process on the plurality of first images,

the image processing unit generates a fourth image by executing the noise reduction process on the plurality of second images, and

the image processing unit executes a process different from the noise reduction process on the third image and the fourth image.

6. The endoscope device according to claim 3, wherein the image processing unit calculates three-dimensional (3D) coordinates of at least one point on a surface of the subject on the basis of the first image and the second image.

7. The endoscope device according to claim 5, wherein the image processing unit calculates 3D coordinates of at least one point on a surface of the subject on the basis of the third image and the fourth image.

8. The endoscope device according to claim 6, further comprising a data generation unit,

wherein the illumination light is white light,

the imaging device includes a red color filter, a green color filter, and a blue color filter,

the imaging device generates a color image as the image of the subject,

the color image has information indicating each of brightness of red, brightness of green, and brightness of blue, and

the data generation unit generates data in which 3D coordinates of a plurality of points on the surface of the subject are associated with the information corresponding to the plurality of points.

9. The endoscope device according to claim 6,

wherein the observation optical system includes a first optical system and a second optical system,

the first optical system and the second optical system are disposed on an optical front side of the imaging device,

the first optical system forms a first optical image of the subject corresponding to a first viewpoint on the imaging device,

the second optical system forms a second optical image of the subject corresponding to a second viewpoint different from the first viewpoint on the imaging device,

the switching unit causes light that passes through the first optical system to be incident on the imaging device and blocks light that passes through the second optical system under the first imaging condition,

the switching unit causes light that passes through the second optical system to be incident on the imaging device and blocks light that passes through the first optical system under the second imaging condition,

the control unit switches the optical image to be formed on the imaging device between the first optical image and the second optical image by controlling the switching unit, and

the image processing unit calculates the 3D coordinates using a passive stereo method on the basis of the first image corresponding to the first optical image and the second image corresponding to the second optical image.

10. The endoscope device according to claim 1, further comprising:

an image processing unit configured to execute image processing on a plurality of images during each frame period of the plurality of frame periods;

a data generation unit;

a first light source serving as the light source; and

a second light source serving as the light source,

wherein the illumination light includes first illumination light and second illumination light,

the first light source generates white light serving as the first illumination light,

the second light source generates the second illumination light,

the illumination optical system includes a pattern generation unit configured to give a spatial pattern including a bright part and a dark part to the second illumination light,

the illumination optical system radiates the second illumination light to which the pattern is given to the subject,

the plurality of imaging conditions include a first imaging condition and a second imaging condition,

under the first imaging condition, the first illumination light is radiated to the subject and radiation of the second illumination light to the subject is stopped,

under the second imaging condition, the second illumination light is radiated to the subject and radiation of the first illumination light to the subject is stopped,

the imaging device generates a first image of the subject by imaging the subject under the first imaging condition,

the imaging device generates a second image of the subject by imaging the subject under the second imaging condition,

the image processing unit calculates 3D coordinates of a plurality of points on a surface of the subject using an active stereo method on the basis of the second image,

the imaging device includes a red color filter, a green color filter, and a blue color filter,

the imaging device generates a color image as the first image,

the color image has information indicating each of brightness of red, brightness of green, and brightness of blue, and

the data generation unit generates data in which the 3D coordinates of the plurality of points are associated with the information corresponding to the plurality of points.

11. The endoscope device according to claim 10,

wherein the imaging device generates a plurality of at least one of first images and second images,

when the imaging device generates the plurality of first images, the image processing unit calculates a value indicating whether or not the plurality of first images are suitable for image processing, and

when the imaging device generates the plurality of second images, the image processing unit calculates a value indicating whether or not the plurality of second images are suitable for image processing.

12. The endoscope device according to claim 10,

wherein the imaging device generates a plurality of second images by imaging the subject under the second imaging condition,

the image processing unit generates a third image by executing a noise reduction process on the plurality of second images, and

the image processing unit calculates the 3D coordinates of the plurality of points on the basis of the third image.

13. The endoscope device according to claim 3,

wherein the illumination optical system includes a pattern generation unit configured to give a spatial pattern including a bright part and a dark part to the illumination light,

the plurality of imaging conditions further include a third imaging condition,

the imaging device generates a third image of the subject by imaging the subject under the third imaging condition,

the switching unit performs switching between the first imaging condition, the second imaging condition, and the third imaging condition by causing a phase of the pattern of the illumination light to be shifted,

a pattern phase under the first imaging condition, a pattern phase under the second imaging condition, and a pattern phase under the third imaging condition are different from one another, and

the image processing unit calculates 3D coordinates of a plurality of points on a surface of the subject using a phase shift method on the basis of at least the first image, the second image, and the third image.

14. The endoscope device according to claim 13, further comprising:

a data generation unit;

a first light source serving as the light source; and

a second light source serving as the light source,

wherein the illumination light includes first illumination light and second illumination light,

the first light source generates white light serving as the first illumination light,

the second light source generates the second illumination light,

the pattern generation unit gives the pattern to the second illumination light,

the plurality of imaging conditions further include a fourth imaging condition,

under the first imaging condition, the second imaging condition, and the third imaging condition, the second illumination light is radiated to the subject and radiation of the first illumination light to the subject is stopped,

under the fourth imaging condition, the first illumination light is radiated to the subject and radiation of the second illumination light to the subject is stopped,

the imaging device generates a fourth image of the subject by imaging the subject under the fourth imaging condition,

the imaging device includes a red color filter, a green color filter, and a blue color filter,

the imaging device generates a color image as the fourth image,

the color image has information indicating each of brightness of red, brightness of green, and brightness of blue, and

the data generation unit generates data in which the 3D coordinates of the plurality of points on the surface of the subject are associated with the information corresponding to the plurality of points.

15. The endoscope device according to claim 14,

wherein the imaging device generates a plurality of at least one of first images, second images, third images, and fourth images,

when the imaging device generates the plurality of first images, the image processing unit calculates a value indicating whether or not the plurality of first images are suitable for image processing,

when the imaging device generates the plurality of second images, the image processing unit calculates a value indicating whether or not the plurality of second images are suitable for image processing,

when the imaging device generates the plurality of third images, the image processing unit calculates a value indicating whether or not the plurality of third images are suitable for image processing, and

when the imaging device generates the plurality of fourth images, the image processing unit calculates a value indicating whether or not the plurality of fourth images are suitable for image processing.

16. The endoscope device according to claim 13,

wherein the control unit causes the switching unit to set the first imaging condition during a plurality of first frame periods,

the control unit causes the switching unit to set the second imaging condition during a plurality of second frame periods and each second frame period of the plurality of second frame periods is different from each first frame period of the plurality of first frame periods,

the control unit causes the switching unit to set the third imaging condition during a plurality of third frame periods and each third frame period of the plurality of third frame periods is different from each first frame period of the plurality of first frame periods and is different from each of second frame period of the plurality of second frame periods,

the image processing unit generates a fifth image by executing a noise reduction process on a plurality of first images,

the image processing unit generates a sixth image by executing the noise reduction process on a plurality of second images,

the image processing unit generates a seventh image by executing the noise reduction process on a plurality of third images, and

the image processing unit calculates the 3D coordinates of the plurality of points on the basis of the fifth image, the sixth image, and the seventh image.

17. The endoscope device according to claim 3,

wherein a focus of the observation optical system under the first imaging condition is different from a focus of the observation optical system under the second imaging condition, and

the image processing unit generates a third image by synthesizing the first image and the second image.

18. The endoscope device according to claim 3,

wherein the amount of light of the light source under the first imaging condition is different from the amount of light of the light source under the second imaging condition, and

the image processing unit generates a third image by synthesizing the first image and the second image.

19. The endoscope device according to claim 3,

wherein the observation optical system includes

a first optical system disposed on an optical front side of the imaging device and configured to form a first optical image of the subject on the imaging device,

a second optical system disposed on an optical front side of the imaging device and configured to form a second optical image of the subject on the imaging device, and

the switching unit configured to select either one of the first optical system and the second optical system and cause only either one of the first optical image and the second optical image to be formed on the imaging device,

a visual field of the first optical system and a visual field of the second optical system have a common region,

the switching control unit switches the optical image formed on the imaging device by controlling the switching unit,

the first optical image is formed on the imaging device under the first imaging condition,

the second optical image is formed on the imaging device under the second imaging condition, and

the image processing unit aligns a region of the first image corresponding to the common region and a region of the second image corresponding to the common region and generates a third image by synthesizing the first image and the second image.

20. The endoscope device according to claim 3,

wherein a focus of the observation optical system under the first imaging condition is different from a focus of the observation optical system under the second imaging condition,

the control unit causes the switching unit to set the first imaging condition during a plurality of first frame periods,

the control unit causes the switching unit to set the second imaging condition during a plurality of second frame periods and each second frame period of the plurality of second frame periods is different from each first frame period of the plurality of first frame periods,

the image processing unit generates a fourth image by executing a noise reduction process on a plurality of first images,

the image processing unit generates a fifth image by executing the noise reduction process on a plurality of second images, and

the image processing unit generates the third image by synthesizing the fourth image and the fifth image.