Method of processing endoscopic images

• US 4,895,431 A
• Filed: 11/10/1987
• Issued: 01/23/1990
• Est. Priority Date: 11/13/1986
• Status: Expired due to Term

• Current Assignee: Olympus Optical Company Ltd. (Olympus Corporation)
• Original Assignee: Olympus Optical Company Ltd. (Olympus Corporation)
• Inventors: Kikuchi, Susumu, Ohyama, Nagaaki, Honda, Toshio, Badique, Eric, Tsujiuchi, Junpei
• Primary Examiner(s): Arnold, Bruce Y.
• Assistant Examiner(s): Ben, Loha
• Application Number:
  US07/119,784
• Time in Patent Office: 805 Days
• Field of Search: 350/320, 350/321, 350/3.77, 350/3.68, 350/3.82, 350/162.12, 350/162.13, 356/347, 356/390, 356/397, 356/398, 128/4, 128/303.15, 358/125, 342/64, 364/822
• US Class Current: 359/29
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/0051 with controlled bending of insertion part
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61M 25/0133 Tip steering devices
• CPC Class Codes: G02B 23/2484 Arrangements in relation to a camera or imaging device processing or control of video signals generated by an endoscope H04N23/60, H04N23/70
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: Y10S 359/90 Methods

Abstract

A method of processing images taken by an endoscope having insertion section, bending section and operation section including a step of entering a first image of an object taken by the endoscope having the bending section situated in a first position, a step of entering a second image taken by the endoscope having the bending section situated in a second position, after moving the bending section into the second position by operating an operation member, the first image being partially overlapped with the second image, a step of detecting a positional relation between the first and second positions by detecting the movement of the bending section, a step of deriving a distance between corresponding points on the first and second images, a step of deriving three dimensional information of the object, and a step of displaying a three dimensional image of the object in accordance with the three dimensional information.

Claims

1. A method of processing endoscope images taken by an endoscope having an insertion section, a bending section provided at a distal end of the insertion section and an operation section for controlling a movement of the bending section, comprising the steps of:

entering a first endoscopic image of an object taken by the endoscope situated in a first position;

entering a second endoscopic image of the object taken by the endoscope situated in a second position;

said second endoscopic image being at least partially overlapped with said first endoscopic image; and

detecting a positional relation between said first and second positions of the endoscope.

2. A method according to claim 1, wherein said first and second endoscopic images are taken by moving the bending section of the endoscope.

3. A method according to claim 2, wherein said bending section of the endoscope is moved by actuating an operation member provided in the operation section which is provided at a proximal end of the insertion section of the endoscope, and said positional relation is detected by detecting a movement of said operation member.

4. A method according to claim 3, wherein said bending section is moved by rotating an angle handle provided in the operation section, said angle handle being coupled with said bending section via an angle wire, and the positional relation is detected by detecting a rotation of said angle handle.

5. A method according to claim 4, wherein said positional relation is detected by detecting a rotation of a pattern disc coupled coaxially with the angle handle.

6. A method according to claim 5, wherein the rotation of said pattern disc is detected by a photoelectric sensor.

7. A method according to claim 4, wherein said positional relation is detected by detecting the movement of the angle wire on which a pattern is provided.

8. A method according to claim 2, wherein said bending section is bent at a single position such that a direction of an optical axis of an objective lens provided in a distal end of the bending section with respect to the object is changed.

9. A method according to claim 2, wherein said bending section is bent at two positions such that a direction of an optical axis of an objective lens provided in a distal end of the bending section with respect to the object is remained unchanged.

10. A method of processing endoscopic images taken by an endoscope having an insertion section, a bending section provided at a distal end of the insertion section, and an operation section for controlling a movement of the bending section, comprising the steps of:

entering a first endoscopic image of an object taken by the endoscope in a first position;

entering a second endoscopic image of the object taken by the endoscope in a second position, said second endoscopic image being at least partially overlapped with said first endoscopic image;

detecting a positional relation between said first and second endoscopic images; and

processing said first and second endoscopic images in accordance with said positional relation to derive geometric information of the object.

11. A method according to claim 10, wherein said processing step comprises a step of deriving a shift amount represented by a distance between corresponding two points on the first and second endoscopic images.

12. A method according to claim 11, wherein said distance is detected by deriving a correlation between small regions in the first and second endoscopic images.

13. A method according to claim 12, wherein prior to deriving the correlation, distortion of the first and second endoscopic images is corrected.

14. A method according to claim 13, wherein after the distortion of the endoscopic images has been corrected, the endoscopic images are subjected to an interpolation.

15. A method according to claim 14, wherein said interpolation is carried out in accordance with a b-spline function.

16. A method according to claim 12, wherein said correlation is detected with the aid of an electronic correlation calculator.

17. A method according to claim 16, wherein said step of detecting the correlation comprises a step of deriving correlation values of first regions in the first and second endoscopic images, a step of selecting first regions whose correlation values exceed a predetermined threshold value, a step of deriving correlation values of second regions which are smaller than said first regions in selected first regions, a step of detecting a second region having the maximum correlation value, and a step of deriving an address of the second region having the maximum correlation value.

18. A method according to claim 16, wherein said correlation is calculated in accordance with the following equation, ##EQU13## wherein f·

g is an average of a product of the first and second images f and g, e,ovs/f/ ·

e,ovs/g/ is a product of an average of f and a average of g, and σ

_f and σ

_g are standard deviations of the images f and g.

19. A method according to claim 12, wherein prior to deriving the correlation, the first and second endoscopic images are subjected to a spatial frequency filtering.

20. A method according to claim 12, wherein each of said first and second endoscopic images is subjected to Fourier transformation and spatial frequency filtering, successively, and output signals derived by the spatial frequency filtering are multiplied, and a multiplied signal is subjected to inverse Fourier transformation.

21. A method according to claim 19 or 20, wherein said spatial frequency filtering is carried out by extracting high frequency components of the first and second endoscopic images.

22. A method according to claim 21 wherein the high frequency components are extracted with the aid of a Laplacian filter.

23. A method according to claim 19 or 20, wherein said spatial frequency filtering is effected by extracting low frequency components in the first and second endoscopic images.

24. A method according to claim 23, wherein the low frequency components are extracted with the aid of an averaging filter.

25. A method according to claim 16, wherein the correlation value C is derived by the following equation;

##EQU14## wherein F^-1 represents the inverse Fourier transformation, F and G* are Fourier transformations of the first and second endoscopic images and |F| and |G| are absolute values of F and G*.

26. A method according to claim 12, wherein said correlation is detected with the aid of an optical correlation calculator.

27. A method according to claim 26, wherein the step of detecting the distance comprises producing a holographic film containing a part of the first endoscopic image, forming a superimposed image of an image of a part of said holographic film and of an image of a part of the second endoscopic image, and detecting a position of a point having maximum light intensity in said superimposed image.

28. A method according to claim 27, wherein said holographic film is produced by projecting an image of said part of the first endoscopic image borne on a photographic film onto a raw photographic film together with reference light, and by developing the raw photographic film to provide said holographic film.

29. A method according to claim 27, wherein said holographic film is produced by displaying an incoherent image of the first endoscopic image on a display device, converting the incoherent image into a coherent image, projecting a part of the coherent image onto a raw photographic film together with reference light, and developing the raw photographic film to provide said holographic film.

30. A method according to claim 11, wherein said processing step comprises a step of calculating a height of a relevant point of the object in accordance with the shift amount.

31. A method according to claim 11, wherein said processing step comprises a step of calculating a distance between two points on the object in accordance with the shift amount.

32. A method according to claim 10, further comprising a step of displaying the first and second endoscopic images, a gray image representing depressions and protrusions of the object, and a three dimensional image of the object.

33. A method according to claim 32, wherein said displaying step comprises displaying a distance between two points on a displayed image, said points being indicated by cursors.

34. A method according to claim 32, wherein said displaying step comprises displaying a height of a point on a displayed image, said point being denoted by a cursor.