Still image stabilization suitable for compact camera environments

• US 20080030587A1
• Filed: 08/07/2006
• Published: 02/07/2008
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Inventors: Helbing, Rene
• Granted Patent: US 8,068,140 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode

Abstract

Still image apparatus and methods that are suitable for compact camera environments are described. In another aspect, image segments are captured during corresponding exposure intervals of a frame exposure period. Respective motion sensor signals are generated during the respective exposure intervals of the image segments. One or more corresponding movement measures are determined from the respective motion sensor signals for each of the exposure intervals. The image segments are combined into an output image based on the one or more corresponding movement measures.

Claims

1. An apparatus, comprising:

an image sensor operable to capture image segments during corresponding exposure intervals of a frame exposure period;

a motion sensor operable to generate respective motion sensor signals during the respective exposure intervals of the image segments; and

a processing system operable to determine from the respective motion sensor signals one or more corresponding movement measures for each of the exposure intervals and combine the image segments into an output image based on the one or more corresponding movement measures.

2. The apparatus of claim 1, wherein the image sensor comprises an array of pixels arranged in rows and columns in a capture plane and the image sensor captures each of the image segments by reading out values of the pixels across a respective subset of the rows of pixels integrated over the corresponding exposure interval.

3. The apparatus of claim 2, further comprising an exposure controller operable to control the capture of the image segments by the image sensor in accordance with a rolling shutter capture process.

4. The apparatus of claim 2, wherein the processing system combines the image segments by deblurring each of the image segments based on the one or more corresponding movement measures and arranging the deblurred image segments along respective rows of the output image.

5. The apparatus of claim 1, wherein the image sensor comprises an array of pixels arranged in rows and columns in a capture plane and the image sensor captures each of the image segments by reading out values of the pixels across the capture plane integrated over the corresponding exposure interval.

6. The apparatus of claim 5, further comprising an exposure controller operable to control the capture of image segments by the image sensor in accordance with a divided exposure capture process.

7. The apparatus of claim 5, wherein the processing system combines the image segments by warping the image segments to a common reference frame based on the one or more corresponding movement measures and combining values of corresponding pixels in the warped image segments into corresponding values of pixels in the output image.

8. The apparatus of claim 1, wherein the motion sensor comprises an inertial sensor operable to generate the motion sensor signals in response to movement of the inertial sensor relative to an inertial reference frame, and the processing system is operable to determine the movement measures by integrating the motion sensor signals generated by the inertial sensor.

9. The apparatus of claim 1, wherein the motion sensor comprises image sensing elements operable to capture images of a scene, and the processing system is operable to determine the movement measures from comparisons of ones of the images captured by the image sensing elements of the motion sensor.

10. The apparatus of claim 1, further comprising a telephone subsystem and a housing containing the image sensor, the motion sensor, the processing system, and the telephone subsystem.

11. A method, comprising:

capturing image segments during corresponding exposure intervals of a frame exposure period;

generating respective motion sensor signals during the respective exposure intervals of the image segments;

determining from the respective motion sensor signals one or more corresponding movement measures for each of the exposure intervals; and

combining the image segments into an output image based on the one or more corresponding movement measures.

12. The method of claim 11, wherein the capturing comprises capturing each of the image segments by reading out values of pixels across a respective subset of rows of a two-dimensional array of pixels integrated over the corresponding exposure interval.

13. The method of claim 12, wherein the capturing of the image segments is performed in accordance with a rolling shutter capture process.

14. The method of claim 12, wherein the combining comprises deblurring each of the image segments based on the one or more corresponding movement measures and arranging the deblurred image segments along respective rows of the output image.

15. The method of claim 11, wherein the capturing comprises capturing each of the image segments by reading out values of pixels across a two-dimensional array of pixels integrated over the corresponding exposure interval.

16. The method of claim 15, wherein the capturing is performed in accordance with a divided exposure capture process.

17. The method of claim 15, wherein the combining comprises warping the image segments to a common reference frame based on the one or more corresponding movement measures and combining values of corresponding pixels in the warped image segments into corresponding values of pixels in the output image.

18. The method of claim 11, wherein the generating comprises generating the motion sensor signals in response to movement relative to an inertial reference frame, and the determining comprises integrating the motion sensor signals.

19. The method of claim 11, wherein the generating comprises capturing images of a scene, and the determining comprises deriving the movement measures from comparisons of ones of the captured images.

20. An apparatus, comprising:

means for capturing image segments during corresponding exposure intervals of a frame exposure period;

means for generating respective motion sensor signals during the respective exposure intervals of the image segments;

means for determining from the respective motion sensor signals one or more corresponding movement measures for each of the exposure intervals; and

means for combining the image segments into an output image based on the one or more corresponding movement measures.