Method and system for motion estimation in digital imaging applications

• US 8,120,659 B2
• Filed: 05/22/2008
• Issued: 02/21/2012
• Est. Priority Date: 05/22/2008
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Li, Dong, Lin, Sheng
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): Berhan, Ahmed A
• Application Number:
  US12/125,425
• Publication Number:
  US 20090290029A1
• Time in Patent Office: 1,370 Days
• Field of Search: 348/207.99, 348/208.99, 348/169, 348/154, 348/155, 348/208.4, 382/107
• US Class Current: 348/208.4
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method and system for estimating global motion in digital imaging applications with confidence level filtering. Motion search yields a plurality of vectors describing the movement of feature blocks from one frame to another. A confidence level is assigned to each motion vector. Based on the confidence levels, vectors are given more or less weight when estimating global motion. High-confidence vectors are given more weight than low-confidence vectors. Embodiments for estimating both the magnitude of global motion and the magnitude and direction of global motion are disclosed.

Claims

1. A method of estimating global motion magnitude in digital video frames, the method comprising:

identifying for each of a plurality of features in the digital video frames a best motion vector among a plurality of candidate motion vectors that most closely corresponds to the motion of the feature between a first frame and a second frame and a second best motion vector that second most closely corresponds to the motion of the feature between the first frame and the second frame;

assigning a confidence level to each best vector based on a comparison of the best motion vector with a respective second best motion vector;

calculating a magnitude of each best motion vector;

determining an average motion magnitude based on the magnitude and confidence level of the best motion vectors; and

estimating the global motion magnitude based on the magnitude and confidence level of each best motion vector having magnitudes within a window of magnitudes surrounding the average motion magnitude, wherein the best and second best motion vectors satisfy the following condition;

|m_min2−

m_min|+|n_min2−

n_min|>

1

wherein m_min, is the horizontal component of the best motion vector, n_min is the vertical component of the best motion vector, m_min2 is the horizontal component of the second best motion vector, and n_min2 is the vertical component of the second best motion vector.

2. The method of claim 1, wherein the identifying step comprises computing a sum of absolute differences (SAD) associated with each of the candidate motion vectors.

3. The method of claim 2, wherein the identifying step further comprises selecting the motion vector with the lowest SAD as a best motion vector and selecting the motion vector with the next lowest SAD as a second best motion vector.

4. The method of claim 3, where the confidence level is assigned as follows:

5. A method of estimating global motion magnitude in digital video frames, the method comprising:

identifying for each of a plurality of features in the digital video frames a best motion vector among a plurality of candidate motion vectors that most closely corresponds to the motion of the feature between a first frame and a second frame and a second best motion vector that second most closely corresponds to the motion of the feature between the first frame and the second frame;

assigning a confidence level to each best vector based on a comparison of the best motion vector with a respective second best motion vector;

calculating a magnitude of each best motion vector;

determining an average motion magnitude based on the magnitude and confidence level of the best motion vectors; and

estimating the global motion magnitude based on the magnitude and confidence level of each best motion vector having magnitudes within a window of magnitudes surrounding the average motion magnitude, wherein the average motion magnitude is calculated as follows;

6. A method of estimating global motion magnitude in digital video frames, the method comprising:

identifying for each of a plurality of features in the digital video frames a best motion vector among a plurality of candidate motion vectors that most closely corresponds to the motion of the feature between a first frame and a second frame and a second best motion vector that second most closely corresponds to the motion of the feature between the first frame and the second frame;

assigning a confidence level to each best vector based on a comparison of the best motion vector with a respective second best motion vector;

calculating a magnitude of each best motion vector;

determining an average motion magnitude based on the magnitude and confidence level of the best motion vectors; and

estimating the global motion magnitude based on the magnitude and confidence level of each best motion vector having magnitudes within a window of magnitudes surrounding the average motion magnitude, wherein a magnitude of a best motion vector is determined to be within the window if the following condition is true;

7. A method of estimating global motion magnitude in digital video frames, the method comprising:

identifying for each of a plurality of features in the digital video frames a best motion vector among a plurality of candidate motion vectors that most closely corresponds to the motion of the feature between a first frame and a second frame and a second best motion vector that second most closely corresponds to the motion of the feature between the first frame and the second frame;

assigning a confidence level to each best vector based on a comparison of the best motion vector with a respective second best motion vector;

calculating a magnitude of each best motion vector;

determining an average motion magnitude based on the magnitude and confidence level of the best motion vectors; and

estimating the global motion magnitude based on the magnitude and confidence level of each best motion vector having magnitudes within a window of magnitudes surrounding the average motion magnitude, wherein the global motion magnitude is estimated as follows;

8. A method of estimating a global motion vector in digital video frames, the method comprising:

identifying for each of a plurality of feature blocks a first motion vector that most closely corresponds to the motion of the feature block between a first frame and a second frame and a second motion vector that second most closely corresponds to the motion of the feature block between the first frame and the second frame;

computing a first sum of absolute differences associated with the first motion vector and a second sum of absolute differences associated with the second motion vector;

assigning a confidence level to each first motion vector based on a comparison of the second sum of absolute differences with the first sum of absolute differences multiplied by an empirically-determined constant;

assigning a weight to each first motion vector based on the location of the first motion vector within the first or second frame;

calculating a weighted distance of each first motion vector;

selecting a subset of the first motion vectors having lowest weighted distances among the first motion vectors; and

estimating the global motion vector based on the subset of first motion vectors and the confidence level assigned to each first motion vector in the subset, wherein the weighted distance of a motion vector is calculated as follows;

9. The method of claim 8, wherein γ

=1 and the norm of a vector V is calculated as follows;

∥

V∥

₁=|x|+|y|

wherein x is the horizontal component of the vector V and γ

is the vertical component of the vector V.

10. A method of estimating a global motion vector in digital video frames, the method comprising:

identifying for each of a plurality of feature blocks a first motion vector that most closely corresponds to the motion of the feature block between a first frame and a second frame and a second motion vector that second most closely corresponds to the motion of the feature block between the first frame and the second frame;

computing a first sum of absolute differences associated with the first motion vector and a second sum of absolute differences associated with the second motion vector;

assigning a confidence level to each first motion vector based on a comparison of the second sum of absolute differences with the first sum of absolute differences multiplied by an empirically-determined constant;

assigning a weight to each first motion vector based on the location of the first motion vector within the first or second frame;

calculating a weighted distance of each first motion vector;

selecting a subset of the first motion vectors having lowest weighted distances among the first motion vectors; and

estimating the global motion vector based on the subset of first motion vectors and the confidence level assigned to each first motion vector in the subset, wherein the number of first motion vectors in the subset is calculated as follows;

11. A method of estimating a global motion vector in digital video frames, the method comprising:

identifying for each of a plurality of feature blocks a first motion vector that most closely corresponds to the motion of the feature block between a first frame and a second frame and a second motion vector that second most closely corresponds to the motion of the feature block between the first frame and the second frame;

computing a first sum of absolute differences associated with the first motion vector and a second sum of absolute differences associated with the second motion vector;

assigning a confidence level to each first motion vector based on a comparison of the second sum of absolute differences with the first sum of absolute differences multiplied by an empirically-determined constant;

assigning a weight to each first motion vector based on the location of the first motion vector within the first or second frame;

calculating a weighted distance of each first motion vector;

selecting a subset of the first motion vectors having lowest weighted distances among the first motion vectors; and

estimating the global motion vector based on the subset of first motion vectors and the confidence level assigned to each first motion vector in the subset, wherein the global motion vector is estimated as follows;