Motion stabilization in video frames using motion vectors and reliability blocks

• US 7,649,549 B2
• Filed: 09/22/2005
• Issued: 01/19/2010
• Est. Priority Date: 09/27/2004
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Batur, Aziz Umit
• Primary Examiner(s): Chan; Jason
• Assistant Examiner(s): Giles; Nicholas G
• Application Number:
  US11/233,445
• Publication Number:
  US 20060066728A1
• Time in Patent Office: 1,580 Days
• Field of Search: 348/208.1, 348/208.99, 348/208.4, 348/208.6
• US Class Current: 348/208.6
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

Stabilization for devices such as hand-held camcorders segments a low-resolution frame into a region of reliable estimation, finds a global motion vector for the region at high resolution, and uses the global motion vector to compensate for jitter.

Claims

1. A method of a digital signal processor for stabilizing a video sequence, comprising:

(a) providing a low resolution version of an input picture;

(b) segmenting said low resolution version into reliable motion estimation blocks and unreliable motion estimation blocks;

(c) finding a single motion vector for the aggregation of said reliable motion estimation blocks;

(d) finding a global motion vector for said input picture from said single motion vector by scaling and refining said single motion vector, wherein said scaling and refining includes a first scaling by a factor of 2 in both horizontal and vertical directions, a first single motion vector refinement by a local search, a second scaling by a factor of 2 in both horizontal and vertical directions, and second motion vector refinement by a second local search;

(e) compensating for jitter motion in said input picture using said global motion vector.

2. The method of claim 1 further comprising combining said reliable motion estimation blocks into a large block.

3. A method of jitter estimation for video frames, comprising:

(a) providing a low resolution version of an input frame;

(b) decomposing said low resolution frame into blocks;

(c) for each of said blocks, computing motion vector prediction errors and when an average motion vector prediction error exceeds a minimum motion vector prediction error by a first threshold, designating said each of said blocks as a reliable block;

(d) for each of said blocks, computing a relative motion vector using a

corresponding relative motion vector from a prior low resolution frame;

(e) taking integer N equal to the number of said reliable blocks with a relative motion vector which differs from the average of said relative motion vectors of all of said reliable blocks by more than a second threshold;

(f) when said N is larger than a third threshold, for each of said reliable blocks which has a relative motion vector greater than a fourth threshold, changing the designation from reliable to unreliable;

(g) finding a motion vector of an aggregate of said reliable blocks;

(h) extending said motion vector of step (g) to a global motion vector of a region in said input frame corresponding to said aggregate in said low resolution frame; and

(i) applying said global motion vector to stabilize said input frame.

4. The method of claim 3, further comprising prior to said step (g), for each of said reliable blocks which has a motion vector magnitude larger than the fourth threshold, changing the designation from reliable to unreliable.

5. The method of claim 3, further comprising, terminating the method when the average of said relative motion vectors has magnitude larger than at least one of the thresholds.

6. The method of claim 3, wherein said applying step includes an adaptive accumulation of said global motion vector to define a displacement.

7. A video camera, comprising:

(a) means for providing a low resolution version of an input frame;

(b) means for decomposing said low resolution frame into blocks;

(c) for each of said blocks, means for computing motion vector prediction errors and when an average motion vector prediction error exceeds a minimum motion vector prediction error by a first threshold, means for designating said each of said blocks as a reliable block;

(d) for each of said blocks, means for computing a relative motion vector using a corresponding relative motion vector from a prior low resolution frame;

(e) means for taking integer N equal to the number of said reliable blocks with a relative motion vector which differs from the average of said relative motion vectors of all of said reliable blocks by more than a second threshold;

(f) when said N is larger than a third threshold, for each of said reliable blocks which has a relative motion vector greater than a fourth threshold, means for changing the designation from reliable to unreliable;

(g) means for finding a motion vector of an aggregate of said reliable blocks;

(h) means for extending said motion vector of step (g) to a global motion vector of a region in said input frame corresponding to said aggregate in said low resolution frame; and

(i) means for applying said global motion vector to stabilize said input frame.