MOTION STABILIZATION

• US 20060244836A1
• Filed: 04/24/2006
• Published: 11/02/2006
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Batur, Aziz Umit
• Granted Patent: US 7,605,845 B2
• Time in Patent Office: Days
• US Class Current: 348/208.100
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 7/0132 the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction

Abstract

Stabilization for devices such as hand-held camcoders segments a low-resolution frame into a region of reliable estimation, refines the motion vectors of that region hierarchically while at the same time updating the segmentation, finds a global motion vector for the region at high resolution, and uses the global motion vector to compensate for jitter.

Claims

1. A method of digital image stabilization, comprising the steps of:

(a) providing a low-resolution version of an input digital picture;

(a) partitioning said low-resolution version into reliable blocks and unreliable blocks according to motion prediction errors;

(b) aggregating said reliable blocks into candidate motion regions according to block motion vectors for said reliable blocks;

(c) finding a segmentation of said low-resolution version from said candidate motion regions;

(d) updating said segmentation to a segmentation of said input picture;

(d) finding a global motion vector for said input picture from said segmentation of said input picture; and

(d) compensating for jitter motion in said input picture using said global motion vector.

2. The method of claim 1, wherein said finding a segmentation includes comparing scores for said candidate motion regions where said score for a region includes (i) a difference of an average relative block motion from a maximum, (ii) a measure of the number of said reliable blocks in a region, and (iii) a measure of the number of said reliable blocks in a region which are also in a segmentation for a low-resolution version of a prior picture.

3. The method of claim 1, wherein said partitioning includes for each of said blocks comparing to a threshold the sum of differences of said prediction error for said block and prediction errors with motion vectors equal to said block motion vector plus an increment or decrement of 1 in one of the two components of said block motion vector.

4. The method of claim 1, wherein said updating said segmentation includes a first updating to a segmentation of a middle-resolution version of said input picture plus a second updating of said segmentation of a middle-resolution version to said segmentation of said input picture where said middle-resolution version has (i) higher resolution than said low-resolution version and (ii) lower resolution than said input picture.

5. A method of jitter compensation for video frames, comprising the steps of:

(a) providing a resolution hierarchy, F₁, F₂, . . . F_N, for an input frame with N an integer greater than 2 and said input frame equal to F_N;

(b) partitioning said F₁ into blocks plus border regions;

(c) for each of said blocks, (i) computing a block motion vector, (ii) computing prediction errors for said block motion vector and for motion vectors differing by 1 from said block motion vector, and (iii) when a sum of differences of said prediction errors exceeds a first threshold, designating said each of said blocks as a reliable block;

(d) aggregating said reliable blocks into candidate motion regions according to block motion vectors for said reliable blocks;

(e) selecting a segmentation for F₁ from said candidate motion regions by comparing said regions according to (i) the number of blocks, (ii) the number of blocks corresponding to blocks in a segmentation of a hierarchy of a prior frame, and (iii) an average of relative block motion vectors;

(f) for n=2, . . . , N, selecting a segmentation for F_n by repeating steps (c)-(e) for blocks corresponding to blocks of said segmentation of F_n-1;

(g) computing a global motion vector from said segmentation of F_N; and

(h) using said global motion vector to jitter compensate said input frame.

6. The method of claim 5, wherein said relative block motion vector is a lowpass filtering of said block motion vector.

7. The method of claim 5, wherein said jitter compensate includes an adaptive accumulation of said global motion vector to define a location for a subwindow in said input frame.

8. The method of claim 5, wherein said partitioning into blocks gives between nine and forty blocks.

9. A video camera, comprising:

(a) picture capturer;

(b) a jitter estimator coupled to said picture capturer, said jitter estimator including;

(i) a resolution hierarchy downsampler;

(ii) a segmenter coupled to said downsampler, said segmenter operable to sequentially find a segmentation plus motion vector for each level of a resolution hierarchy computed from an input picture;

(iii) a global motion vector determiner coupled to said segmenter;

(c) a memory for a resolution hierarchy of a prior frame, said memory coupled to said jitter estimator; and

(d) a jitter motion compensater coupled to said jitter estimator.

10. The video camera of claim 9, wherein:

(a) said jitter estimator is implemented as a program on a programmable processor.