Motion stabilization

• US 20060066728A1
• Filed: 09/22/2005
• Published: 03/30/2006
• Est. Priority Date: 09/27/2004
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Batur, Aziz Umit
• Granted Patent: US 7,649,549 B2
• Time in Patent Office: Days
• US Class Current: 348/208.990
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

Stabilization for devices such as hand-held camcoders segments a low-resolution frame into a region of reliable estimation, finds a global motion vector for the region at high resolution, and uses the global motion vector to compensate for jitter.

Claims

1. A method of stabilization for a video sequence, comprising:

(a) provide a low resolution version of an input picture;

(a) segment said low resolution version into reliable motion estimation blocks and unreliable motion estimation blocks;

(b) find a single motion vector for the aggregation of said reliable motion estimation blocks;

(c) find a global motion vector for said input picture from said single motion vector;

(d) compensate for jitter motion in said input picture using said global motion vector.

2. The method of claim 1, wherein:

(a) said low resolution version of an input picture includes at least 9 motion estimation blocks.

3. The method of claim 1, wherein:

(a) said global motion vector is found in step (c) of claim 1 by scaling and refining said single motion vector.

4. The method of claim 3, wherein:

(a) said scaling and refining of claim 3 includes a first scaling by a factor of 2 in both horizontal and vertical directions, a first single motion vector refinement by a local search, a second scaling by a factor of 2 in both horizontal and vertical directions, and second motion vector refinement by a second local search.

5. A method of jitter estimation for video frames, comprising:

(a) provide a low resolution version of an input frame;

(b) decompose said low resolution frame into blocks;

(c) for each of said blocks, compute motion vector prediction errors and when an average motion vector prediction error exceeds a minimum motion vector prediction error by a first threshold, designate said each of said blocks as a reliable block;

(d) for each of said blocks, compute a relative motion vector using a corresponding relative motion vector from a prior low resolution frame;

(e) take integer N equal to the number of said reliable blocks with a relative motion vector which differs from the average of said relative motion vectors of all of said reliable blocks by more than a second threshold;

(f) when said N is larger than a third threshold, for each of said reliable blocks which has a relative motion vector greater than a fourth threshold, change the designation from reliable to unreliable;

(g) find a motion vector of an aggregate of said reliable blocks;

(h) extend said motion vector of step (g) to a global motion vector of a region in said input frame corresponding to said aggregate in said low resolution frame; and

(i) apply said global motion vector to stabilize said input frame.

6. The method of claim 5, further comprising:

(a) prior to said step (g) of claim 5, for each of said reliable blocks which has a motion vector magnitude larger than a fourth threshold, change the designation from reliable to unreliable.

7. The method of claim 5, further comprising:

(a) after step (f) of claim 5, when the average of said relative motion vectors has magnitude larger than a threshold, terminate.

8. The method of claim 5, wherein:

(a) said apply of step (i) of claim 5 includes an adaptive accumulation of said global motion vector to define a displacement.

9. A video camera, comprising:

(a) picture capture circuitry;

(b) a jitter estimator coupled to said picture capture circuitry, said jitter estimator including;

(i) a low-pass filter and downsampler;

(ii) segmentation circuitry coupled to said downsampler, said seqmentation circuitry operable to segment a low resolution version of an input picture into reliable motion estimation blocks and unreliable motion estimation blocks and to find a single motion vector for said reliable motion estimation blocks;

(iii) refinement circuitry coupled to said segmentation circuitry, said refinement circuitry operable to compute a global motion vector for said input picture from said single motion vector; and

(c) jitter motion compensation circuitry coupled to said jitter estimator.

10. The video camera of claim 9, wherein:

(a) said jitter estimator is implemented as a program on a programmable processor.