Endoscopic vision system

• US 8,911,358 B2
• Filed: 07/09/2007
• Issued: 12/16/2014
• Est. Priority Date: 07/10/2006
• Status: Expired due to Fees

• Current Assignee: Katholieke Universiteit Leuven
• Original Assignee: Katholieke Universiteit Leuven
• Inventors: Koninckx, Philippe, Koninckx, Thomas, Van Gool, Luc
• Primary Examiner(s): Nguyen, Anhtuan T
• Assistant Examiner(s): CHOU, WILLIAM B
• Application Number:
  US12/373,036
• Publication Number:
  US 20090259102A1
• Time in Patent Office: 2,717 Days
• Field of Search: 348/47, 348/48, 382/294, 600/103, 600/111, 600/117, 600/166
• US Class Current: 600/113
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00181 for multiple fixed viewing angles A61B1/00193 takes precedence
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes

Abstract

A multiple viewpoint endoscope system having a multiple viewpoint camera setup and/or an intelligent or cognitive image control system and display device particularly adapted for localizing internal structures within a cavity or an enclosing structure, such as an animal body, for instance the abdomen of an animal or human, or for localizing a real or synthetic image of such internal structures within an overview image or on an overview 3D model.

Claims

1. An endoscopic or borescopic vision system for inspecting an inside of a cavity or guiding operation on enclosed structures in said cavity, comprising:

an imaging system to receive selected images of a target object and to create output signals of the target object image;

the imaging system comprising a camera system arranged to create output signals of different object images of portions of the target object structure or a direct environment thereof, the camera system comprising a first ocular system and at least a second ocular system, at deployment or in operational working state ocular distances between the first and second ocular systems or relative orientation of the first and second ocular systems arranged to be controllable to change different viewpoints relative to each other;

the camera system comprising a first and a second camera or multiple secondary cameras designed to create said output signals of different object images of portions of the target object structure or the direct environment thereof based on multiple viewpoint wide baseline imaging, and further comprising an image processing or control arrangement, communicatively coupled to an image acquiring circuitry and configured to process the received signals of the target object image, which cannot be seen by the first camera, and which is under normal working situations not possible with a stereo system,

the image processing arrangement being arranged to find corresponding image points between the images from different cameras based on a wide baseline matching system and comprising a computer program code arranged to carry out a turnkey technique to find corresponding regions of object images from separate cameras and to direct the cameras of the imaging system towards a shared working volume,

the ocular systems having re-orientable camera heads configured to automatically fine-tune a field of view of the camera heads,

wherein a baseline between the first and second ocular systems is arranged to be selectable at deployment or in operational working state in the range 20 to 250 mm, the distance between the ocular systems is arranged to be changeable and wherein the imaging system is configured to determine a configuration or constellation of the camera system fine-tuned using a cognitive control unit and combine the output signals into integrated video data taking into account said configuration or constellation of the camera system, the integrated video data comprising a synthetic overview image, in which generating the synthetic overview image comprises using texture blending with illumination correction on a depth modulated projection surface.

2. The endoscopic or borescopic vision system of claim 1, wherein the distances between the first and second ocular systems are configured to be changeable but wherein at deployment or in an operational working state, the ocular distance is above 50 mm, resulting in a wide baseline vision system, given the size of structures in a scene being imaged during endoscopy.

3. The vision system according to claim 1, the camera system comprising a multi view camera system configured to generate different viewpoints by each of the cameras or to provide several viewpoints of the target object each by different cameras, such that the cameras have a different zoom and orientation.

4. The vision system according to claim 1, wherein the first camera is a principal camera configured to create an object image of the target object on the inside of the cavity to be operated and the second camera or multiple secondary cameras is or are secondary camera or cameras configured to provide an image of another portion of the object.

5. The vision system according to claim 1, wherein the image processing arrangement is configured to transform multiple video streams of the cameras into a synthetic wide angle overview image data in real time and present the image data to the user, and to generate one or more virtual camera images, without the need for motion of any optical components.

6. The vision system according to claim 1, wherein the image processing arrangement is configured to extract 3D information from multiple images of which some are obtainable from a camera that is not mounted to the imaging system.

7. The vision system according to claim 1, wherein the image processing arrangement is configured to generate 3D information based on a projection of structured light through a regular unmodified mono focal endoscope, while imaging the scene through a multi-view endoscopic camera network.

8. The vision system according to claim 1, wherein the image processing arrangement is configured to generate 3D information based on a projection of structured light through a stereo endoscope, while imaging a scene through one optical channel of the endoscope, and projecting through the other optical channel.

9. The vision system according to claim 1, wherein the image processing arrangement is configured to implement a disaster recovery procedure or a safety procedure using an underlying model or synthetic overview image as a good approximation of the target image.

10. The vision system according to claim 1, wherein the image processing arrangement is configured to implement enhanced reality visualization merging endoscopic video data with data coming from another modality based on an interpreted scene data by a cognitive processing arrangement.

11. The vision system of claim 10, wherein said cognitive processing arrangement is configured to use local or distributed intelligence to merge the video data based on scene interpretation.

12. The vision system of claim 1, wherein the secondary camera or multiple secondary cameras are inserted into the cavity using endoscopic or borescopic lenses through secondary entry ports.

13. The vision system of claim 1, wherein the secondary cameras are inserted into the cavity using a flexible endoscopic or flexible borescopic through secondary entry ports.

14. The endoscopic vision system comprising two or more cameras of claim 1 for use in a laparoscopic operation.