Motion stabilization

• US 7,605,845 B2
• Filed: 04/24/2006
• Issued: 10/20/2009
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Batur, Aziz Umit
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): Giles; Nicholas G
• Application Number:
  US11/379,835
• Publication Number:
  US 20060244836A1
• Time in Patent Office: 1,275 Days
• Field of Search: 348/208.1, 348/208.99, 348/208.4, 348/208.6, 375/240.16
• US Class Current: 348/208.6
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 7/0132 the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction

Abstract

Stabilization for devices such as hand-held camcorders segments a low-resolution frame into a region of reliable estimation, refines the motion vectors of that region hierarchically while at the same time updating the segmentation, finds a global motion vector for the region at high resolution, and uses the global motion vector to compensate for jitter.

Claims

1. A method of digital image stabilization, comprising the steps of:

providing a low-resolution version of an input digital picture;

partitioning said low-resolution version into reliable blocks and unreliable blocks according to motion prediction errors;

aggregating said reliable blocks into candidate motion regions according to block motion vectors for said reliable blocks;

finding a segmentation of said low-resolution version from said candidate motion regions, wherein said finding a segmentation includes comparing scores for said candidate motion regions where said score for a region includes a difference of an average relative block motion from a maximum, a measure of the number of said reliable blocks in a region, and a measure of the number of said reliable blocks in a region which are also in a segmentation for a low-resolution version of a prior picture;

updating said segmentation to a segmentation of said input picture;

finding a global motion vector for said input picture from said segmentation of said input picture; and

compensating for jitter motion in said input picture using said global motion vector.

2. The method of claim 1, wherein said updating said segmentation includes a first updating to a segmentation of a middle-resolution version of said input picture plus a second updating of said segmentation of a middle-resolution version to said segmentation of said input picture where said middle-resolution version has (i) higher resolution than said low-resolution version and (ii) lower resolution than said input picture.

3. A method of jitter compensation for video frames, comprising the steps of:

providing a resolution hierarchy, F₁, F₂, . . . F_N, for an input frame with N an integer greater than 2 and said input frame equal to F_N;

partitioning said F₁ into blocks plus border regions;

for each of said blocks, (i) computing a block motion vector, (ii) computing prediction errors for said block motion vector and for motion vectors differing by 1 from said block motion vector, and (iii) when a sum of differences of said prediction errors exceeds a first threshold, designating said each of said blocks as a reliable block;

aggregating said reliable blocks into candidate motion regions according to block motion vectors for said reliable blocks;

selecting a segmentation for F₁ from said candidate motion regions by comparing said regions according to (i) the number of blocks, (ii) the number of blocks corresponding to blocks in a segmentation of a hierarchy of a prior frame, and (iii) an average of relative block motion vectors, wherein said segmentation comprises comparing scores for candidate motion regions where said score for a region includes a difference of an average relative block motion from a maximum, a measure of the number of said reliable blocks in a region, and a measure of the number of said reliable blocks in a region which are also in a segmentation for a low-resolution version of a prior picture;

for n=2, . . . , N, selecting a segmentation for F_n by repeating the above steps for blocks corresponding to blocks of said segmentation of F<sub>n−

1</sub>;

computing a global motion vector from said segmentation of F_N; and

using said global motion vector to jitter compensate said input frame.

4. The method of claim 3, wherein said relative block motion vector is a lowpass filtering of said block motion vector.

5. The method of claim 3, wherein said jitter compensate includes an adaptive accumulation of said global motion vector to define a location for a subwindow in said input frame.

6. The method of claim 3, wherein said partitioning into blocks gives between nine and forty blocks.

7. A video camera, comprising:

picture capturer;

a jitter estimator coupled to said picture capturer, said jitter estimator including;

a resolution hierarchy downsampler;

a segmenter coupled to said downsampler, said segmenter operable to sequentially find a segmentation plus motion vector for each level of a resolution hierarchy computed from an input picture, wherein said segmentation comprises comparing scores for candidate motion regions where said score for a region includes a difference of an average relative block motion from a maximum, a measure of the number of said reliable blocks in a region, and a measure of the number of said reliable blocks in a region which are also in a segmentation for a low-resolution version of a prior picture;

a global motion vector determiner coupled to said segmenter;

a memory for a resolution hierarchy of a prior frame, said memory coupled to said jitter estimator; and

jitter motion compensater coupled to said jitter estimator.