Panoramic imaging and display system with canonical magnifier

• US 20030103063A1
• Filed: 12/03/2001
• Published: 06/05/2003
• Est. Priority Date: 12/03/2001
• Status: Active Grant

• Current Assignee: Biotronix Inc.
• Original Assignee: Tempest Microsystems
• Inventors: Branson, Steven, Mojaver, Michael
• Granted Patent: US 6,833,843 B2
• Time in Patent Office: Days
• US Class Current: 345/647
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

The present invention provides an image acquisition and viewing system that employs a fish-eye lens and an imager, such as, a CCD array, to obtain a wide angle image, e.g., an image of a hemispherical field of view. The system of the invention further includes a primary display for displaying the image, and a secondary display for presenting a perspective-corrected view of a selected portion of the image. A processor effects correction of a selected portion of the image for distortions introduced by the fish-eye lens by utilizing methods according to the teachings of the invention. The processor further effects the display of the image and its perspective-corrected portion thereof on the primary and the secondary displays, respectively. Moreover, the processor effects the display of a graphical object on the first display to delineate the portion that is presented on the secondary display. This allows a viewer to view simultaneously the image and the perspective-corrected portion, thereby gleaning information regarding the context of the perspective-corrected portion within the entire image while viewing it.

Claims

1. A device for providing a perspective corrected view of at least a portion of a wide angle image, comprising:

an image-capturing device having a fish-eye lens for generating the wide angle image,

a processor coupled to the image capturing device that receives the image, the processor further comprising;

a first display module for displaying the wide angle image, and

a second display module for displaying a perspective corrected view of a portion of the image,

the processor further comprising a correlation module for displaying a graphical object on the displayed wide angle image that identifies the selected portion of the image displayed in the perspective corrected view.

2. The device of claim 1, wherein the processor further comprising a scale adjuster that generates a perspective corrected view of the portion of the image at a user-adjustable magnification.

3. The device of claim 1, further comprising a user interface module coupled to the processor for selecting a portion of the image to be displayed as a perspective corrected view.

4. The device of claim 3, wherein the user interface allows selecting a magnification for viewing the perspective-corrected portion of the image.

5. The device of claim 1, wherein the wide angle image is a fish-eye image.

6. The device of claim 2, wherein the processor generates the perspective corrected view of a portion of the image by transforming that portion according to the defined magnification and an angle for viewing a section of the hemisphere corresponding to that image portion from a point offset from the center of the hemisphere by a defined distance.

7. The device of claim 5, wherein the image-capturing device further comprises an imager coupled to the fish-eye lens for converting optical photons collected by the lens from the field of view into electrical signals representing an image of the field of view.

8. The device of claim 6, wherein the imager can be any of a CCD array, a CMOS array, or a thermal imaging device.

9. The device of claim 7, wherein the imager has a resolution in a range of about one million to about 100 million effective pixels per square inch.

10. The device of claim 5, wherein the processor generates the perspective-corrected view by mapping a point (u,v) on an undistorted image plane corresponding to a perspective-corrected portion of a distorted image to a point (x,y) on a plane corresponding to the distorted image, the mapping is provided by the following equations:

x=R(β

₀/(PI/2))cos(d₀) y=R(β

₀/(PI/2))sin(d₀) t=[Dd+sqrt(D²d²−

(u²+v²+d²)(D²−

1))]/(u²+v²+d²) β

₀=arctan(−

D cos(β

)+dt cos(β

)−

vt sin(β

),

     1) δ

₀=arctan(−

D sin(β

)+dt sin(β

)+vt cos(β

), ut)+δ

where β and

δ

are the zenith and azimuthal angles corresponding to the center of the undistorted image, β

₀ and δ

₀ are the zenith and azimuthal angles corresponding to a pixel (u,v) in the undistorted image, d is a magnification factor, D is a level of undistortion, and R is the radius of the fisheye image.

11. The device of claim 10, wherein each of the distorted and undistorted image planes includes a two-dimensional array of pixels providing a digital luminance map corresponding to the image or a perspective-corrected portion of the image, respectively.

12. The device of claim 11, wherein the processor generates the luminance value of a pixel corresponding to the point (u,v) by calculating a weighted average of luminance values of two or more pixels surrounding the point (x,y), if the point (x,y) does not correspond to a pixel on the distorted image plane.

13. A device for providing a perspective corrected view of at least a portion of a wide angle image, comprising:

an image-capturing device having a fish-eye lens for generating the wide angle image,

a processor coupled to the image capturing device that receives the image,

the processor displaying the image and a graphical object associated with the image for selecting a portion thereof,

wherein the processor responds to the selection of a portion of the image by generating a perspective corrected view of that portion and displaying the perspective corrected view.

14. The device of claim 13, further comprising a user interface module coupled to the graphical object and the processor for effecting the selection of a portion of the image by associating the graphical object with that portion.

15. The device of claim 14, wherein the user interface module permits selecting a magnification for viewing the portion of the image selected by the graphical object.

16. The device of claim 15, wherein the processor corrects the selected portion of the image for perspective distortions in accord with the selected magnification and an angle for viewing a portion of a hemispherical field of view corresponding to the selected image portion from a vantage point offset from a center of the hemisphere.

17. A device for providing a perspective-corrected view of at least a portion of a wide angle image, comprising:

an image-capturing device having a fish-eye lens for generating the wide angle image,

a processor in communication with the image-capturing device,

the processor correcting at least a portion of the image for distortions introduced by the fish-eye lens by mapping a point (u,v) on an undistorted image plane corresponding to a perspective-corrected view of the image portion to a point (x,y) on a distorted image plane corresponding to the imag in accord with an angle for viewing a section of the hemisphere corresponding to the image portion from a vantage point offset from a center of the hemisphere and a distance between the vantage point and the center of the hemisphere.

18. The device of claim 17, wherein the processor employs the following equations to effect the mapping between the point (u,v) and the point (x,y):

x=R(β

₀/(PI/2))cos(d₀) y=R(β

₀/(PI/2))sin(d₀) t=[Dd+sqrt(D²d²−

(u²+v²+d²)(D²−

1))]/(u²+v²+d²) β

₀=arctan(−

D cos(β

)+dt cos(β

)−

vt sin(β

),

     1) δ

₀=arctan(−

D sin(β

)+dt sin(β

)+vt cos(β

), ut)+δ

where β and

δ

are the zenith and azimuthal angles corresponding to the center of the undistorted image, β

₀ and δ

₀ are the zenith and azimuthal angles corresponding to a pixel (u,v) in the undistorted image, d is the magnification factor, D is the level of undistortion, and R is the radius of the fisheye image

19. A device for imaging a field of view, comprising:

an image-capturing device having a fish-eye lens for generating a wide angle image of the field of view,

a processor coupled to the image-capturing device to receive the image, the processor displaying the wide-angle image and further selecting a portion of the image based on a pre-programmed set of rules,

the processor generating a perspective corrected view of the selected portion and displaying the perspective-corrected view.

20. The device of claim 19, further comprising a buffer for storing the set of rules.

21. The device of claim 20, wherein the processor identifies the selected portion on the image presented on the first display.

22. The device of claim 21, wherein the processor displays a graphical object on the displayed wide-angle image to identify a portion of the image corresponding to the perspective-corrected view.

23. A device for imaging a field of view, comprising

an image-capturing device having a fish-eye lens for acquiring a fish-eye image of the field of view,

a processor in communication with the image-capturing device,

a display coupled to the processor for presenting the fish-eye image and a perspective-corrected view of a portion thereof,

a graphical object presented on the display in association with a portion of the fish-eye image,

the processor effecting the presentation of the fish-eye image on the display, and further generating a perspective-corrected view of the portion of the fish-eye image associated with the graphical object, and effecting the display of the perspective-corrected view in an area of the display circumscribed by the graphical object.

24. A method for generating a perspective-corrected view of a portion of a fish-eye image, comprising:

mapping a point (u,v) on an undistorted image plane corresponding to a perspective view of the image portion to a point (x,y) on a distorted image plane corresponding to the fish-eye image according to the following equations;

x=R(β

₀/(PI/2))cos(d₀) y=R(β

₀/(PI/2))sin(d₀) t=[Dd+sqrt(D²d²−

(u²+v²+d²)(D²−

1))]/(u²+v²+d²) β

₀=arctan(−

D cos(β

)+dt cos(β

)−

vt sin(β

),

     1) δ

₀=arctan(−

D sin(β

)+dt sin(β

)+vt cos(β

), ut)+δ

where β and

δ

are the zenith and azimuthal angles corresponding to the center of the undistorted image, β

₀ and δ

₀ are the zenith and azimuthal angles corresponding to a pixel (u,v) in the undistorted image, d is the magnification factor, D is the level of undistortion, and R is the radius of the fisheye image.