Panoramic Camera With Multiple Image Sensors Using Timed Shutters

• US 20090201361A1
• Filed: 02/09/2009
• Published: 08/13/2009
• Est. Priority Date: 02/08/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Holt, Jason, McClatchie, Iain Richard Tyrone, Embler, Gary, Lyon, Richard F.
• Granted Patent: US 8,493,436 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS

Abstract

The present invention relates to the field of panoramic still and motion photography. In a first embodiment, a camera apparatus for panoramic photography includes a first image sensor positioned to capture a first image. The first image sensor has a rolling-shutter readout arranged in portrait orientation. The camera apparatus also includes second image sensor positioned to capture a second image. The second image sensor has a rolling-shutter readout arranged in portrait orientation. Finally, the camera apparatus includes a controller configured to signal the second image sensor to start capturing the second image before the first image sensor finishes capturing the first image. At least a portion of the first image is in front of the second image relative to a forward direction of the camera apparatus.

Claims

1. A camera apparatus for panoramic photography, comprising:

a first image sensor positioned to capture a first image, the first image sensor having a rolling-shutter readout arranged in portrait orientation;

a second image sensor positioned to capture a second image, the second image sensor having a rolling-shutter readout arranged in portrait orientation; and

a controller configured to signal the second image sensor to start capturing the second image before the first image sensor finishes capturing the first image,

wherein at least a portion of the first image is in front of the second image relative to a forward direction of the camera apparatus.

2. The camera apparatus of claim 1, wherein the controller is configured to signal the second image sensor to start capturing the second image approximately when the first image sensor starts to capture an overlap of the fields of view of the first and second image sensor.

3. The camera apparatus of claim 1, wherein the controller is configured to signal the second image sensor to start capturing the second image at a delay time corresponding to an overlap of the fields of view of the first and second image sensors.

4. The camera apparatus of claim 3, wherein the delay time is offset according to a velocity value of the camera apparatus to use motion parallax resulting from motion of the camera apparatus to cancel out spatial parallax between the first and second image sensors.

5. The camera apparatus of claim 4, wherein the velocity value of the camera apparatus is an approximate or average velocity of the camera apparatus.

6. The camera apparatus of claim 4, wherein the delay time is offset by a time between when an entrance pupil of the first image sensor passes a location and when an entrance pupil of the second image sensor passes the location during forward motion of the camera apparatus.

7. The camera apparatus of claim 4, wherein the delay time approximately satisfies the following equation:

$t=R_1·<br><br>F_1-\frac{D_{12}}{v},$

wherein t is the delay time,

wherein R₁ is a rolling-shutter readout time of the first image sensor,

wherein F₁ is a percentage of the field of view of the first image sensor which does not overlap with the field of view of the second image sensor,

wherein D₁₂ is a distance between an entrance pupil of the first image sensor and an entrance pupil of the second image sensor, and

wherein v is a speed of the camera apparatus.

8. The camera apparatus of claim 1, wherein the first and second image sensors are attached to a vehicle.

9. The camera apparatus of claim 1, wherein the first image sensor exposes at least a first and second column of the first image sensor, the first column capturing a more rearward portion of the first image and the second column capturing a more forward portion of the first image relative to forward motion of the camera apparatus,

wherein the first column of the first image sensor starts exposure before the second column of the first image sensor,

wherein the second image sensor exposes at least a first and second column of the second image sensor, the first column capturing a more rearward portion of the second image and the second column capturing a more forward portion of the second image relative to the forward motion of the camera apparatus, and

wherein the first column of the second image sensor starts exposure before the second column of the second image sensor.

10. The camera apparatus of claim 1, further comprising at least six additional image sensors, wherein a computer is able to stitch together images taken from the at least eight total cameras to create a panoramic image.

11. The camera apparatus of claim 1, wherein the first and second image sensors are CMOS image sensors.

12. A method for panoramic photography, comprising:

(a) capturing a first image with a first image sensor, the first image sensor having a rolling-shutter readout arranged in landscape orientation; and

(b) starting to capture a second image with a second image sensor prior to completion of the capturing in (a), the second image sensor having a rolling-shutter readout arranged in landscape orientation,

wherein at least a portion of the first image is in front of the second image relative to a forward direction of a camera apparatus comprising the first and second image sensors.

13. The method of claim 12, wherein the starting to capture (b) occurs at a delay time corresponding to an overlap of the fields of view of the first and second image sensors.

14. The method of claim 13, wherein the delay time is offset according to a velocity value of the camera apparatus to use motion parallax resulting from motion of the camera apparatus to cancel out spatial parallax between the first and second image sensors.

15. A camera apparatus for motion photography, comprising:

a first camera having a first entrance pupil wherein, during forward motion of the camera apparatus, the first camera captures a first image; and

a second camera having a second entrance pupil wherein, during the forward motion of the camera apparatus, the second camera captures a second image,

wherein motion of the camera apparatus results in motion parallax, and

wherein the timing of when the first camera captures a first image relative to when the second camera captures the second image uses the motion parallax to reduce the effect of spatial parallax between the first camera and the second camera.

16. The camera apparatus of claim 15, wherein, during forward motion of the camera apparatus, the first camera captures a first image when the first entrance pupil is at a first location and the second camera captures a second image when the second entrance pupil is at a second location, wherein the second location is approximately equal to the first location.

17. The camera apparatus of claim 15, wherein the first and second cameras are attached to a vehicle.

18. The camera apparatus of claim 15, further comprising at least six additional image sensors, wherein a computer is able to stitch together images taken from the at least eight total cameras to create a panoramic image.

19. A method for motion photography with a camera apparatus, comprising:

(a) capturing a first image with a first camera in the camera apparatus at a first time during forward motion of the camera apparatus; and

(b) capturing a second image with a second camera in the camera apparatus at a second time during forward motion of the camera apparatus,

wherein motion of the camera apparatus results in motion parallax,

wherein the timing of when the capturing (a) occurs relative to when the capturing (b) occurs uses the motion parallax to reduce the effect of spatial parallax between the first camera and the second camera.

20. The method of claim 19, wherein the capturing the first image (a) comprises capturing the first image when a first entrance pupil of the first camera is at a first location, and

wherein the capturing the second image (b) comprises capturing the second image when the second entrance pupil is at a second location, wherein the second location is approximately equal to the first location.