Method and apparatus for panoramic imaging

• US 7,834,910 B2
• Filed: 03/01/2006
• Issued: 11/16/2010
• Est. Priority Date: 03/01/2006
• Status: Active Grant

• Current Assignee: Garmin Switzerland GmbH (Garmin Limited)
• Original Assignee: David M. Delorme
• Inventors: DeLorme, David M., Peterson, James G.
• Primary Examiner(s): SELBY, GEVELL V
• Application Number:
  US11/365,141
• Publication Number:
  US 20070206945A1
• Time in Patent Office: 1,721 Days
• Field of Search: 348 36- 39, 348/148, 348/218.1, 348/222.1, 348/294
• US Class Current: 348/218.1
• CPC Class Codes: G03B 41/00 Special techniques not covered by groups G03B31/00 - G03B39/00; Apparatus therefor
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Camera system and methods to capture panoramic imagery from a camera mounted on a moving platform, using low-cost digital image sensors. The panoramic imagery appears seamless and natural to the eye. The panoramic imaging system and methods are specifically designed to accommodate the long acquisition times of low-cost digital image sensors, despite the motion of the camera during image capture. Pairs of cameras are arranged about an axis and a pairwise firing sequence enables capturing a series of adjacent images without gap or overlap. Additionally, when combined with suitable supplemental sensors, the image data provide location information about objects in the image for use in elementary photogrammetry.

Claims

1. A panoramic imaging system for capturing a full or partial panoramic image, said camera comprising:

a compound camera having an orientation axis extending through said compound camera from a first axis end to a second axis end, said compound camera comprising a plurality of cameras adapted to take a plurality of images, each single camera of said plurality of cameras having an image sensor for sensing an image, said image sensor having rows, wherein a predetermined row in said image sensor is defined as row one and said image sensor is oriented so as to scan said image with vertical scan lines, starting with said row one, and wherein said image sensor of said each single camera is oriented such that said row one is closer to said first axis end than any other row in said image sensor;

a memory means for storage of said plurality of images; and

a controller for synchronizing a firing of said plurality of cameras according to a pre-defined firing sequence and directing storage of said plurality of images in said memory means;

wherein said firing sequence first fires a first camera on a first side of said orientation axis, and then fires a second camera that is adjacent said first camera on said first side of said orientation axis, and wherein said each single camera has a field of view that is sufficiently wide that said plurality of images obtained in said firing sequence forms a contiguous panoramic image without a gap between any two images captured by adjacent cameras of said plurality of cameras.

2. The panoramic imaging system of claim 1, wherein said image sensor is without memory capability, wherein said memory means is adapted to receive image data from said image sensor.

3. The panoramic imaging system of claim 1, wherein said plurality of cameras includes a pair of cameras that has a first camera and a second camera, said first camera and said second camera arranged in mirror locations on each side of said orientation axis.

4. The panoramic imaging system of claim 3 further comprising a controller for synchronizing a firing of said plurality of cameras according to a pre-defined firing sequence and directing storage of said plurality of images in said memory means, wherein said plurality of cameras includes at least a first pair and a second pair of cameras, said second pair of cameras arranged adjacent said first pair of cameras, a first camera of said first pair and of said second pair being positioned to a first side of said orientation axis and a second camera of said first pair and of said second pair being positioned on a second side of said orientation axis, and wherein said firing sequence simultaneously fires said first camera and said second camera of said first pair, and then simultaneously fires said first camera and said second camera of said second pair.

5. The panoramic imaging system of claim 1, wherein said plurality of cameras includes two or more pairs of cameras.

6. The panoramic imaging system of claim 1, further comprising supplemental sensor means.

7. The panoramic imaging system of claim 6, wherein said supplemental sensor means includes a speed sensing device.

8. The panoramic imaging system of claim 6, wherein said supplemental sensor means includes a direction sensing device.

9. The panoramic imaging system of claim 6, wherein said supplemental sensor means includes an altitude sensing device.

10. The panoramic imaging system of claim 6, wherein said supplemental sensor means includes a position sensing device.

11. The panoramic imaging system of claim 6, wherein said supplemental sensor means includes sensors for collecting data on ambient conditions.

12. The panoramic imaging system of claim 1 further comprising a mass storage means for receiving and storing data from said memory means.

13. A method of capturing and processing images with a panoramic imaging system, so as to minimize the effects of motion, said method comprising steps of:

(a) arranging a plurality of pairs of cameras that form a compound camera about an orientation axis, said orientation axis extending through said compound camera from a first axis end to a second axis end, each single camera of a pair of cameras being placed in mirror locations on opposite sides of said orientation axis, and each camera of said plurality of cameras having an image sensor, wherein said image sensor in each said single camera is oriented such that a pre-defined first row is oriented vertically and is closer to said first axis end than any other row in said image sensor;

(b) firing said pairs of cameras according to a pre-defined firing sequence, one pair of cameras at a time;

(c) capturing a first image from said each camera during said firing by scanning sequentially across said image sensor, row by row, wherein said scanning generates a plurality of vertical scan lines; and

(d) processing said first image of said each single camera to create an undistorted composite panoramic view.

14. The method of claim 13, wherein said processing further comprises the step of:

(e) correcting said first image for motion-related distortion.

15. The method of claim 14, said step (e) including the steps:

(e1) determining a travel distance that said compound camera has traveled over an image acquisition period defined by a first instant and a second instant;

(e2) determining an initial first distance line between an object captured by said image sensor and said image sensor at said first instant;

(e3) determining a second distance line between said object and said image sensor at said second instant and determining an angle between said first distance line and said second distance line; and

(e4) calculating said approximate correction angle from a function;

A(t)=t+½

[1−

cos(2t)] arctan(D/2R).

16. The method of 15, said processing step further comprising the steps of:

(d1)) applying blending techniques to said first composite panoramic view; and

(d2) reprojecting said first composite panoramic view onto a final cylindrical format to create a composite panoramic view that has no visual discontinuities.