Systems and methods for panoramic imaging

• US 8,817,066 B2
• Filed: 04/22/2011
• Issued: 08/26/2014
• Est. Priority Date: 09/21/2007
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Krishnan, Gurunandan, Nayar, Shree K.
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): ELAHI, SHAN E
• Application Number:
  US13/092,591
• Publication Number:
  US 20110211039A1
• Time in Patent Office: 1,222 Days
• Field of Search: 348/36, 348 37- 39
• US Class Current: 348/36
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: G02B 7/182 for mirrors
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A mirror assembly adapted for use in a panoramic imaging system for capturing a panoramic image includes a mirror for optically coupling to a fisheye lens having a first field of view, the mirror configured for reflecting an image of a second field of view through the fisheye lens. A housing has a first end and a second end, the mirror being secured proximate to the first end, and the second end having an engagement portion for securing the mirror assembly to the panoramic imaging system. When the mirror assembly is secured to the panoramic imaging system, the mirror is optically coupled to the fisheye lens and a detector for capturing a first portion of the panoramic image and a second portion of the panoramic image, the first portion of the panoramic image having a portion overlapping the second portion of the panoramic image.

Claims

1. A mirror assembly adapted for use in a panoramic imaging system for capturing a panoramic image, comprising:

a mirror having a mirror field of view and being for optically coupling to a fisheye lens having a direct field of view, the mirror configured for reflecting an image of the mirror field of view through the fisheye lens;

a housing having a first end and a second end, the mirror being secured proximate to the first end, and the second end having an engagement portion for securing the mirror assembly to the panoramic imaging system,

wherein, when the mirror assembly is secured to the panoramic imaging system, the mirror is optically coupled to the fisheye lens and a detector for capturing a first portion of the panoramic image corresponding to the direct field of view and a second portion of the panoramic image corresponding to the mirror field of view, the first portion of the panoramic image having a portion overlapping the second portion of the panoramic image;

a processor for operatively coupling to the detector; and

the memory storing program instructions, and wherein, when the mirror assembly is secured to the panoramic imaging system, the program instructions being executed by the processor cause the processor to;

receive the captured first and second portions of the panoramic image from the detector,

combine the first and second of the panoramic image to form the panoramic image, and

smooth the overlapping portion of the first and second portions of the panoramic image by feathering.

2. The mirror assembly of claim 1, wherein the mirror is a spherical mirror.

3. The mirror assembly of claim 1, wherein a radius of curvature of the mirror and a width of the mirror are predetermined as a function of a position of the mirror with respect to the fisheye lens.

4. The mirror assembly of claim 1, wherein the housing further comprises a cylindrical transparent wall disposed between the first end and the second end.

5. The mirror assembly of claim 1, wherein the engagement portion comprises a screw thread.

6. The mirror assembly of claim 1, wherein, when the mirror assembly is secured to the panoramic imaging system, the mirror is positioned coaxially with regard to the fisheye lens.

7. The mirror assembly of claim 1, wherein, when the mirror assembly is secured to the panoramic imaging system, a lower angle of a panoramic field of view of the panoramic imaging system is less than or equal to a lower angle of the mirror field of view.

8. The mirror assembly of claim 1, wherein, when the mirror assembly is secured to the panoramic imaging system, an angle of overlap between the direct field of view and the mirror field of view is equal to the difference between a lower angle of the direct field of view and an upper angle of the mirror field of view.

9. The mirror assembly of claim 1, wherein the program instructions being executed by the processor cause the processor to utilize the detector to:

capture the first portion of the panoramic image corresponding to the direct field of view transmitted directly through the fisheye lens, and

capture the second portion of the panoramic image corresponding to the mirror field of view reflected from the mirror through the fisheye lens.

10. The mirror assembly of claim 9, wherein the combining of the first portion of the panoramic image and the second portion of the panoramic image is performed utilizing a scale-invariant feature transform algorithm.

11. The mirror assembly of claim 9, wherein the memory stores program instructions that when executed by the processor, further cause the processor to calibrate the system.

12. The mirror assembly of claim 11, wherein calibrating the system comprises calibrating the mirror by determining the center of curvature and orientation of the mirror.

13. The mirror assembly of claim 9, wherein the memory stores program instructions that, when executed by the processor, further cause the processor generate a table of correspondence between pixel locations on the first portion of the panoramic image and pixel locations on the second portion of the panoramic image.