Panoramic imaging and display system with canonical magnifier

• US 6,833,843 B2
• Filed: 12/03/2001
• Issued: 12/21/2004
• Est. Priority Date: 12/03/2001
• Status: Expired due to Term

• Current Assignee: Biotronix Inc.
• Original Assignee: Tempest Microsystems
• Inventors: Branson, Steven, Mojaver, Michael
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): RICHER, AARON M
• Application Number:
  US10/005,824
• Publication Number:
  US 20030103063A1
• Time in Patent Office: 1,114 Days
• Field of Search: 345/647, 345/427, 348/207.99, 348/144
• US Class Current: 345/647
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

The present invention provides an image acquisition and viewing system that employs a fish-eye lens and an imager, such as, a CCD array, to obtain a wide angle image, e.g., an image of a hemispherical field of view. The system of the invention further includes a primary display for displaying the image, and a secondary display for presenting a perspective-corrected view of a selected portion of the image. A processor effects correction of a selected portion of the image for distortions introduced by the fish-eye lens by utilizing methods according to the teachings of the invention. The processor further effects the display of the image and its perspective-corrected portion thereof on the primary and the secondary displays, respectively. Moreover, the processor effects the display of a graphical object on the first display to delineate the portion that is presented on the secondary display. This allows a viewer to view simultaneously the image and the perspective-corrected portion, thereby gleaning information regarding the context of the perspective-corrected portion within the entire image while viewing it.

Claims

1. A device for providing a perspective-corrected view of at least a portion of a wide angle image, comprising:

an image-capturing device having a fish-eye lens for generating the wide angle image as a projection of at least a section of a viewing hemisphere associated with the lens onto an image plane,

a processor coupled to the image capturing device that receives the image, the processor further comprising;

a first display module for displaying the wide angle image, and

a second display module for displaying a perspective-corrected view of portion of the image,

the processor further comprising a correlation module for displaying a graphical object on the displayed wide angle image that identifies the selected portion of the image displayed in the perspective-corrected view,

wherein the processor generates said perspective-corrected view from a vantage point offset relative to a center of said viewing hemisphere.

2. The device of claim 1, wherein the processor further comprises a scale adjuster that applies to said perspective-corrected view a user-adjustable magnification.

3. The device of claim 2, wherein the processor generates the perspective-corrected view of a portion of the image by transforming that portion according to a defined magnification and an angle for viewing a section of the hemisphere corresponding to that image portion from said offset vantage point.

4. The device of claim 1, further comprising a user interface module coupled to the processor for selecting a portion of the image to be displayed as a perspective corrected view.

5. The device of claim 4, wherein the user interface allows selecting a magnification for viewing the perspective-corrected portion of the image.

6. The device of claim 1, wherein the wide angle image is a fish-eye image.

7. The device of claim 6, wherein the image-capturing device further comprises an imager coupled to the fish-eye lens for converting optical photons collected by the lens from the field of view into electrical signals representing an image of the field of view.

8. The device of claim 7, wherein the imager can be any of a CCD array, a CMOS array, or a thermal imaging device.

9. The device of claim 7, wherein the imager has a resolution in a range of about one million to about 100 million effective pixels per square inch.

10. The device of claim 6, wherein the processor generates the perspective-corrected view by mapping a point (u,v) on an undistorted image plane corresponding to a perspective-corrected portion of a distorted image to a point (x,y) on a plane corresponding to the distorted image, the mapping is provided by the following equations:

11. The device of claim 10, wherein each of the distorted and undistorted image planes includes a two-dimensional array of pixels providing a digital luminance map corresponding to the image or a perspective-corrected portion of the image, respectively.

12. The device of claim 11, wherein the processor generates the luminance value of a pixel corresponding to the point (u,v) by calculating a weighted average of luminance values of two or more pixels surrounding the point (x,y), if the point (x,y) does not correspond to a pixel on the distorted image plane.

13. A device for providing a perspective-corrected view of at least a portion of a wide angle image, comprising:

an image-capturing device having a fish-eye lens for generating the wide angle image,

a processor in communication with the image-capturing device,

the processor correcting at least a portion of the image for distortions introduced by the fish-eye lens by mapping a point (u,v) on an undistorted image plane corresponding to a perspective-corrected view of the image portion to a point (x,y) on a distorted image plane corresponding to the image in accord with an angle for viewing a section of the hemisphere corresponding to the image portion from a vantage point offset from a center of the hemisphere and a distance between the vantage point and the center of the hemisphere.

14. The device of claim 13, wherein the processor employs the following equations to effect the mapping between the point (u,v) and the point (x,y):

15. A method for generating a perspective-corrected view of a portion of a fish-eye image, comprising:

mapping a point (u,v) on an undistorted image plane corresponding to a perspective view of the image portion to a point (x,y) on a distorted image plane corresponding to the fish-eye image according to the following equations;