Omniview motionless camera endoscopy system

• US 5,313,306 A
• Filed: 06/01/1993
• Issued: 05/17/1994
• Est. Priority Date: 05/13/1991
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Telerobotics International, Inc.
• Inventors: Kuban, Daniel P., Martin, H. Lee, Zimmermann, Steven D.
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): Burgess, Glenton B.
• Application Number:
  US08/068,776
• Time in Patent Office: 350 Days
• Field of Search: 358/209, 358/225, 358/98, 358/108, 358/87, 358/85, 358/903, 358/229, 358/180, 358/160, 382/44, 395/137-139
• US Class Current: 348/65
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/00174 characterised by the viewing angles
• CPC Class Codes: A61B 1/00183 for variable viewing angles
• CPC Class Codes: A61B 2090/0813 Accessories designed for easy sterilising, i.e. re-usable
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: H04N 1/2158 using a detachable storage unit
• CPC Class Codes: H04N 1/217 Interfaces allowing access to a single user
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A endoscopic-type device for omnidirectional image viewing providing electronic pan-and-tilt orientation, rotation, and magnification within a selected field-of-view for use in applications in various environments such as in internal medicine inspection, monitoring, and surgery. The imaging device (using optical or infrared images) is based on the effect that the image from a wide angle lens, which produces a circular image of an entire field-of-view, can be mathematically corrected using high speed electronic circuitry. More specifically, an incoming image from a endoscope image acquisition source, including a wide angle lens, is transmitted through an image conduit and captured by a camera that produces output signals according to that image. A transformation of these output signals is performed for the viewing region of interest and viewing direction, and a corrected image is output as a video image signal for viewing, recording, or analysis. Multiple simultaneous images can be output from a single input image.

Claims

1. A device for providing perspective and distortion corrected views, in a desired format, of a selected portion of a field of view in an environment, said device comprising:

an imaging system for receiving selected optical and infrared images of said selected portion of a field of view and for producing output signals corresponding to said selected optical and infrared images, said imaging system including a light source, an image conduit to conduct light from said light source into the environment and return said images from the environment, and a video camera to create said output signals;

a lens attached to a distal end of said image conduit of said imaging system, said distal end for placement in the environment, for conveyance of said selected optical and infrared images through said image conduit of said imaging system;

image capture circuitry for receiving and providing digitized signals from said output signals from said imaging system;

image transform circuitry for processing said digitized signals from said image capture circuitry according to selected viewing angles of pan, tilt, and rotation and percentage of magnification, and for producing output signals according to a combination of said digitized signals and said selected viewing angles of pan, tilt, and rotation and percentage of magnification;

an input for selected user and external computer input to select said angles of pan, tilt, and rotation and percentage of magnification, and for converting said selected viewing angles of pan, tilt, and rotation and percentage of magnification for input to said image transform circuitry to control said processing of said image transform circuitry; and

an output connected to said image transform circuitry for recording said perspective and distortion corrected views according to said selected viewing angles of pan, tilt, and rotation and percentage of magnification.

7. The device of claim 1 wherein said image transform circuitry is programmed to implement the following two equations:

##EQU7## where;

A=(cos φ

cos ∂

-sin φ

sin ∂

cos β

)

B=(sin φ

cos ∂

+cos φ

sin ∂

cos β

)

C=(cos φ

sin ∂

+sin φ

cos ∂

cos β

)

D=(sin φ

sin ∂

-cos φ

cos ∂

cos β

)

and where;

R=radius of the image circle

β

=zenith angle

∂

=Azimuth angle in image plane

φ

=Object plane rotation angle

m=Magnification

u,v=object plane coordinates

x,y=image plane coordinates.

2. An endoscopy device for providing perspective corrected views of a selected portion of perspective distorted optical images from an environment into a desired format, said device comprising:

an imaging system for receiving optical images from the environment and for producing output signals corresponding to said perspective distorted optical images, said imaging system including light source, an image conduit for conveying light from said light source into the environment, a wide angle lens at a distal end of said image conduit in the environment for optical conveyance of said optical images through said image conduit and a video camera positioned exterior the environment to receive images from said lens, said optical images to said video camera being perspective distorted by said wide angle lens;

image capture circuitry for receiving, and producing digitized signals from, said output signals from said imaging system;

image transform circuitry for processing said digitized signals from said image capture circuitry according to selected viewing angles of pan, tilt, and rotation and percentage of magnification for producing output signals of perspective corrected images according to a combination of said digitized signals and said selected viewing angles of pan, tilt, and rotation and percentage of magnification;

an input for selecting said viewing angles of pan, tilt, and rotation and percentage of magnification, and for producing signals to control said processing of said image transform circuitry as to said viewing angles of pan, tilt, and rotation and percentage of magnification; and

an output connected to said image transform circuitry for recording said perspective corrected views according to said selected viewing angles of pan, tilt, and rotation and percentage of magnification.

3. The device of claim 2 wherein said imaging system further comprises a video recorder for receiving and recording said output signals of said video camera for selective input to said image transform circuitry.

4. The device of claim 2 further comprising a further input for inputting control signals to said image transform circuitry to achieve simultaneous multiple perspective corrected views in said output and selected scanning of said perspective corrected views in said output.

5. The device of claim 4 wherein said input and said further input have selective user inputs including computer keyboard input, voice activated input and touch screen input.

6. The device of claim 4 wherein said further input includes discrete switches to control said processing of said image transform circuitry, said discrete switches in a keypad, said keypad being sterilizable when said device is utilized for medical purposes.

8. The device of claim 2 wherein said image transform circuitry is programmed to implement the following two equations:

##EQU8## where;

A=(cos φ

cos ∂

-sin φ

sin ∂

cos β

)

B=(sin φ

cos ∂

+cos φ

sin ∂

cos β

)

C=(cos φ

sin ∂

+sin φ

cos ∂

cos β

)

D=(sin φ

sin ∂

-cos φ

cos ∂

cos β

)

and where;

R=radius of the image circle

β

=zenith angle

∂

=Azimuth angle in image plane

φ

=Object plane rotation angle

m=Magnification

u,v=object plane coordinates

x,y=image plane coordinates.

9. An endoscopic device for use in internal medicine to provide perspective corrected views of a selected portion of perspective distorted optical images from an environment within a patient into a desired format, said device comprising:

an imaging system for receiving optical images from the environment and for producing output signals corresponding to said perspective distorted optical images, said imaging system including an image conduit, a video camera at a proximal end of said image conduit exterior the environment, a wide angle lens at a distal end of said image conduit for placement within the environment for optical conveyance of said optical images from the environment through said image conduit to said video camera, said optical images to said video camera being perspective distorted by said wide angle lens, said imaging system being sterilizable;

image transform circuitry for processing said output signals from said imaging system according to selected viewing angles of pan, tilt, and rotation and percentage of magnification for producing output signals of perspective corrected images according to a combination of said digitized signals and said selected viewing angles of pan, tilt, and rotation and percentage of magnification;

an input for selecting said viewing angles of pan, tilt, and rotation and percentage of magnification, and for producing signals to control said processing of said image transform circuitry as to said viewing angles of pan, tilt, and rotation and percentage of magnification; and

a further input for inputting control signals to said image transform circuitry to achieve multiple perspective corrected views in said output means and selected scanning of said video camera; and

an output connected to said image transform circuitry for recording said perspective corrected views according to said selected viewing angles of pan, tilt, and rotation and percentage of magnification;

wherein said image transform circuitry is programmed to implement the following two equations;

##EQU9## where;

A=(cos φ

cos ∂

-sin φ

sin ∂

cos β

)

B=(sin φ

cos ∂

+cos φ

sin ∂

cos β

)

C=(cos φ

cos ∂

+sin φ

cos ∂

cos β

)

D=(sin φ

sin ∂

-cos φ

cos ∂

cos β

)

and where;

R=radius of the image circle

β

=zenith angle

∂

=Azimuth angle in image plane

φ

=Object plane rotation angle

m=Magnification

u,v=object plane coordinates

x,y=image plane coordinates.

10. The device of claim 9 wherein said input and said further input have selective user inputs including computer keyboard input, voice activated input and touch screen input.

11. The device of claim 10 wherein said further input includes discrete switches to control said processing of said image transform circuitry, said discrete switches in a keypad, said keypad being sterilizable.

12. The device of claim 10 wherein said further input comprises discrete switches to input said selected angles of pan, tilt, and rotation and percentage of magnification into said image transform circuitry.

13. The device of claim 11 further comprising an output line from said image transform circuitry to selectively activate actuating equipment within the environment.

14. The device of claim 9 wherein said imaging system further comprises a light source to direct light through said image conduit into said environment within the patient.

15. An endoscopic device for use in confined industrial inspection to provide perspective corrected views of a selected portion of perspective distorted optical images from an environment within an industrial device into a desired format, said device comprising:

an imaging system for receiving optical images from the environment and for producing output signals corresponding to said perspective distorted optical images, said imaging system including an image conduit, a video camera at a proximal end of said image conduit exterior the environment, a wide angle lens at a distal end of said image conduit for placement within the environment for optical conveyance of said optical images from the environment through said image conduit to said video camera, said optical images to said video camera being perspective distorted by said wide angle lens;

image transform circuitry for processing said output signals from said imaging system according to selected viewing angles of pan, tilt, and rotation and percentage of magnification for producing output signals of perspective corrected images according to a combination of said digitized signals and said selected viewing angles of pan, tilt, and rotation and percentage of magnification;

an input for selecting said viewing angles of pan, tilt, and rotation and percentage of magnification, and for producing signals to control said processing of said image transform circuitry as to said viewing angles of pan, tilt, and rotation and percentage of magnification; and

a further input for inputting control signals to said image transform circuitry to achieve multiple perspective corrected views in said output means and selected scanning of said video camera; and

an output connected to said image transform circuitry for recording said perspective corrected views according to said selected viewing angles of pan, tilt, and rotation and percentage of magnification;

wherein said image transform circuitry is programmed to implement the following two equations;

##EQU10## where;

A=(cos φ

cos ∂

-sin φ

sin ∂

cos β

)

B=(sin φ

cos ∂

+cos φ

sin ∂

cos β

)

C=(cos φ

sin ∂

+cos φ

cos ∂

cos β

)

D=(sin φ

sin ∂

-cos φ

cos ∂

cos β

)

and where;

R=radius of the image circle

β

=zenith angle

∂

=Azimuth angle in image plane

φ

=Object plane rotation angle

m=Magnification

u,v=object plane coordinates

x,y=image plane coordinates.

16. The device of claim 15 wherein said further input includes discrete switches to control said processing of said image transform circuitry, said discrete switches enclosed in a keypad.

17. The device of claim 15 wherein said further input comprises discrete switches to input said selected angles of pan, tilt, and rotation and percentage of magnification into said image transform circuitry.

18. The device of claim 16 further comprising an output line from said image transform circuitry to selectively activate actuating equipment within the environment.

19. The device of claim 15 wherein said imaging system further comprises a light source to direct light through said image conduit into the environment within the industrial device.