Systems, devices and methods for digital camera image stabilization

• US 20080106608A1
• Filed: 11/08/2006
• Published: 05/08/2008
• Est. Priority Date: 11/08/2006
• Status: Active Grant

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: Avago Technologies ECBU IP Singapore Pte Limited (Broadcom, Inc.)
• Inventors: Clark, Airell Richard, Reid, Donald Montgomery, Garvin, Richard Wesley
• Granted Patent: US 7,714,892 B2
• Time in Patent Office: Days
• US Class Current: 348/208.99
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS

Abstract

Disclosed are various systems, devices and methods for digital camera image stabilization. In one embodiment, a digital camera captures a plurality of digital images of a subject simultaneously or near-simultaneously and stores the captured images in memory as individual digital data sets. While the digital images are being captured simultaneously or near-simultaneously by means of a plurality of rolling shutters, a relative position determination device such as an angular rate sensor is employed to sense and save to memory data representative of the relative positions of the digital camera at the moments the individual digital images or frames are being exposed. Spatial shift directions and magnitudes are calculated in a processor of the camera on the basis of such data, and then applied to the individual data sets to form aligned data sets comprising aligned data points. To compensate for camera shake occurring while the individual images were being captured, the aligned data sets are added together to form an output data set representative of an output image.

Claims

1. A digital image stabilization system, comprising:

a sensor pixel array comprising an array of imaging pixels and at least first and second rolling shutters;

a memory containing software;

a processor;

a lens operatively coupled to the sensor pixel array, and

a relative position determination device;

wherein the sensor pixel array, memory, software, processor, lens and relative position determination device are operatively connected to one another and configured to;

(a) capture at least one of simultaneously and near-simultaneously exposed at least first and second images of the same subject, at least portions of the first image being captured by the first rolling shutter while at least portions of the second image are being captured by the second rolling shutter;

(b) spatially align first and second data sets comprising individual data points based on information sensed and provided by the relative position determination device, where the first and second data sets correspond to pixel arrays containing analog pixel voltages, the pixel arrays forming the first and second images, respectively, and (c) add individual data points from the aligned first and second data sets to form an output data set.

2. The digital image stabilization system of claim 1, wherein the system is configured to capture individually underexposed at least first and second images.

3. The digital image stabilization system of claim 1, wherein the system is configured to capture individually properly exposed at least first and second images.

4. The digital image stabilization system of claim 1, wherein the system further comprises at least one analog sample-and-hold register for receiving, storing and transmitting a series of analog voltages corresponding to pixel analog voltages to an analog-to-digital converter or a buffer.

5. The digital image stabilization system of claim 4, wherein the system further comprises at least one analog-to-digital converter located between an output of the at least one analog sample-and-hold register and an input to the memory.

6. The digital image stabilization system of claim 1, wherein the relative position determination device is an angular rate sensor.

7. The digital image stabilization system of claim 1, wherein the relative position determination device is a MEMS device.

8. The digital image stabilization system of claim 1, further comprising means for at least one of white balancing, compensating for slight color shifts under different light conditions, auto-exposure compensating, anti-vignetting, auto black level compensating, auto flicker compensating, color correction compensating, saturation level compensating, color space converting, changing data or pixel format, demosaicking, gamma correcting, halftoning, desaturating, changing pixel color gains, pixel mismatch correcting, pixel sharpening, subsampling, median filtering, spatial filtering, frequency-wavenumber processing, color processing, and color adjusting.

9. The image stabilization system of claim 1, wherein the system is incorporated into at least one of a digital still camera and a digital video camera.

10. The image stabilization system of claim 1, further comprising at least one of a user shutter control, a removable memory, a display, a lens controller and a computer-readable storage medium.

11. A digital image stabilization system, comprising:

means for sensing light and storing same as analog voltages in a pixel array, the sensing and storing means further comprising at least first and second rolling shutter means;

means for storing digital data and software code;

means for processing data and executing the software code;

a lens operatively coupled to the light sensing means, and

means for sensing relative positions;

wherein the light sensing means, the digital data storage means, the software code, the processing means, the lens and the relative position sensing means are operatively connected to one another and configured to;

(a) capture at least one of simultaneously and near-simultaneously exposed at least first and second images of the same subject, at least portions of the first image being captured by the first rolling shutter while at least portions of the second image are being captured by the second rolling shutter;

(b) spatially align first and second data sets comprising individual data points based on information sensed and provided by the relative position determination device, where the first and second data sets correspond to pixel arrays containing analog pixel voltages, the pixel arrays forming the first and second images, respectively, and (c) add individual data points from the aligned first and second data sets to form an output data set.

12. The digital image stabilization system of claim 11, wherein the system is configured to capture individually underexposed at least first and second images.

13. The digital image stabilization system of claim 11, wherein the system is configured to capture individually properly exposed at least first and second images.

14. The digital image stabilization system of claim 11, wherein the system further comprises at least one analog sample-and-hold register for receiving, storing and transmitting a series of analog voltages corresponding to pixel analog voltages to an analog-to-digital converter or a buffer.

15. The digital image stabilization system of claim 14, wherein the system further comprises at least one analog-to-digital converter located between an output of the at least one analog sample-and-hold register and an input to the memory.

16. The digital image stabilization system of claim 11, wherein the relative position sensing means is an angular rate sensor.

17. The digital image stabilization system of claim 11, wherein the relative position sensing means is a MEMS device.

18. The digital image stabilization system of claim 11, further comprising means for at least one of white balancing, compensating for slight color shifts under different light conditions, auto-exposure compensating, anti-vignetting, auto black level compensating, auto flicker compensating, color correction compensating, saturation level compensating, color space converting, changing data or pixel format, demosaicking, gamma correcting, halftoning, desaturating, changing pixel color gains, pixel mismatch correcting, pixel sharpening, subsampling, median filtering, spatial filtering, frequency-wavenumber processing, color processing, and color adjusting.

19. The image stabilization system of claim 11, wherein the system is incorporated into at least one of a digital still camera and a digital video camera.

20. The image stabilization system of claim 11, further comprising at least one of a user shutter control, a removable memory, a display, a lens controller and a computer-readable storage medium.

21. A method of stabilizing digital images in an imaging device, comprising using the device to capture at least one of simultaneously and near-simultaneously a plurality of digital images of the same subject and storing same in a memory thereof as individual data sets, sensing the relative position corresponding to the device for each of the captured images and storing data representative of same in the memory, aligning the individual data sets on the basis of calculations performed using the data, and adding data points from each of the aligned set sets to form an output data set representative of an output image.

22. The method of claim 21, wherein each image is one of underexposed and properly exposed.

23. The method of claim 21, further comprising white balancing, compensating for slight color shifts under different light conditions, auto-exposure compensating, anti-vignetting, auto black level compensating, auto flicker compensating, color correction compensating, saturation level compensating, color space converting, changing data or pixel format, demosaicking, gamma correcting, halftoning, desaturating, changing pixel color gains, pixel mismatch correcting, pixel sharpening, subsampling, median filtering, spatial filtering, frequency-wavenumber processing, color processing, and color adjusting at least one of the individual data sets and the output data set.