METHOD AND SYSTEM FOR MOTION ESTIMATION IN DIGITAL IMAGING APPLICATIONS

• US 20090290029A1
• Filed: 05/22/2008
• Published: 11/26/2009
• Est. Priority Date: 05/22/2008
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Micron Technology, Inc.
• Inventors: Lin, Sheng, Li, Dong
• Granted Patent: US 8,120,659 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method and system for estimating global motion in digital imaging applications with confidence level filtering. Motion search yields a plurality of vectors describing the movement of feature blocks from one frame to another. A confidence level is assigned to each motion vector. Based on the confidence levels, vectors are given more or less weight when estimating global motion. High-confidence vectors are given more weight than low-confidence vectors. Embodiments for estimating both the magnitude of global motion and the magnitude and direction of global motion are disclosed.

Claims

1. A method of processing image data, the method comprising:

generating motion vectors based on detected Notion of feature blocks from a first frame to a second frame;

determining a confidence level associated with each motion vector; and

estimating global motion based on the vectors and respective confidence levels.

5. The method of claim 1, wherein the estimating step comprises:

determining a magnitude for each motion vector;

determining a weighted average of the magnitudes; and

estimating a global motion magnitude based on magnitudes within a window surrounding the weighted average of the magnitudes.

6. The method of claim 5, wherein each motion vector magnitude is weighted based on the confidence level associated with the motion vector.

7. The method of claim 1, wherein the estimating step comprises:

assigning a weight to each motion vector based on the location of the motion vector between the first and second frames;

determining a weighted distance associated with each motion vector; and

estimating a global motion vector based on the weight and weighted distance associated with each motion vector.

8. The method of claim 1, further comprising validating the global motion by comparing a sum of the confidence levels to a threshold.

9. The method of claim 8, further comprising substituting a global motion determined for a prior frame if the sum of the confidence levels is less than the threshold.

10. The method of claim 9, wherein the global motion determined for a prior frame is an average of global motions determined for a plurality of prior frames.

2. The method of claim i, wherein the determining step comprises;

comparing a best motion vector that most accurately describes the detected motion with a second best motion vector that second most accurately describes the detected motion;

comparing the best and second best !notion vectors; and

assigning a confidence level based on a result of the comparison.

3. The method of claim 2, wherein the comparing step comprises comparing a sum of absolute differences associated with the best motion vector to a sum of absolute differences associated with the second best motion vector.

4. The method of claim 3, wherein the confidence level is directly related to the difference between the sums.

29. The imager of claim 2S, wherein the confidence level assigned to each best motion vector is directly related to the difference between the SAD of the best motion vector and the SAD of the second best motion vector.

11. A method of estimating global motion magnitude in digital video frames, the method comprising:

identifying for each of a plurality of features in the digital video frames a best motion vector among a plurality of candidate motion vectors that most closely corresponds to the motion of the feature between and first frame and a second frame and a second best motion vector that second most closely corresponds to the motion of the feature between the first frame and the second frame;

assigning a confidence level to each best vector based on a comparison of the best motion vector with a respective second best motion vector;

calculating a magnitude of each best motion vector;

determining an average motion magnitude based on the magnitude and confidence level of the best motion vectors; and

estimating the global motion magnitude based on the magnitude and confidence level of best motion vectors within a window surrounding the average motion magnitude.

12. The method of claim 11, wherein the identifying step comprises computing a sum of absolute differences (SAD) associated with each of the candidate motion vectors.

13. The method of claim 12, wherein the identifying step further comprises selecting the motion vector with the lowest SAD as the best motion vector and selecting the motion vector with the next lowest SAD as the second best motion vector.

14. The method of claim 13, where the confidence level is assigned as follows:

15. The method of claim 11, wherein the best and second best motion vectors satisfy the following condition:

|m_min2−

m_min|+|n_min2−

n_min|>

1

wherein m_min is the horizontal component of the best motion vector, n_min is the vertical component of the best motion vector, m_min2 is the horizontal component of the second best motion vector, and n_min2 is the vertical component of the second best motion vector.

16. The method of claim 11, wherein the average motion magnitude is calculated as follows:

17. The method of claim 11, wherein a magnitude of a best motion vector is determined to be within the window if the following condition is true:

18. The method of claim 11, wherein the global motion magnitude is estimated as follows:

19. A method of estimating a global motion vector in digital video flames, the method comprising:

identifying for each of a plurality of feature blocks a first motion vector that most closely corresponds to the, motion of the feature block between and first frame and a second frame and a second motion vector that second most closely corresponds to the motion of the feature block between the first frame and the second frame;

assigning a confidence level to each first motion vector based on a difference between the first motion vector and a respective second motion vector;

assigning a weight to each first motion vector based on the location of the first motion vector within the first or second frame;

calculating a weighted distance of each first motion vector;

selecting a subset of the first motion vectors having lowest weighted distances among the first motion vectors; and

estimating the global motion vector based on the subset of first motion vectors and the confidence level assigned to each first motion vector in the subset.

20. The method of claim 19, wherein the weighted distance of a motion vector is calculated as follows:

21. The method of claim 20, wherein γ

=1 and the norm of a vector V is calculated as follows;

∥

V∥

₁=|x|+|y|

wherein x is the horizontal component of the vector V and y is the vertical component of the vector V.

22. The method of claim 19, wherein the number of first motion vectors in the subset is calculated as follows:

23. The method of claim 19, wherein the global motion vector is estimated as follows:

24. An imager, comprising

a pixel array configured to output pixel values comprising first and second images;

a processor configured to determine global motion between the first and second images by;

identifying a plurality of motion vectors corresponding to the translation of respective pixel blocks from the first image to the second image;

assigning a confidence level to each motion vector based on a comparison of a best motion vector that most closely corresponds to the translation of a respective pixel block from the first image to the second image with a second best motion vector that second most closely corresponds to the translation of the respective pixel block; and

estimating global motion based on the motion vectors and respective confidence levels.

25. The imager of claim 24, wherein the processor is further configured to:

assign a weight to each motion vector based on the location of the motion vector within the first or second image,

wherein the extent to which a motion vector influences the global motion vector estimate is based at least in part on the weight assigned to the motion vector.

26. The imager of claim 24, wherein the processor is further configured to identify the plurality of motion vectors by motion searching to identify the best and second best motion vectors associated with each pixel block.

27. The imager of claim 26, wherein the motion searching comprises computing a sum of absolute differences (SAD) associated with the motion vectors.

28. The imager of claim 27, wherein the motion searching further comprises selecting the motion vector with the lowest SAD as the best motion vector and selecting the motion vector with the next lowest SAD as the second best motion vector.

30. The imager of claim 24, wherein the processor is further configured to discard motion vectors that do not satisfy a minimum confidence level.

31. The imager of claim 24, wherein the estimated global motion is a magnitude.

32. The imager of claim 24, wherein the estimated global motion is a vector comprising a horizontal component and a vertical component.

33. A processor system, comprising:

an input device configured to receive frames of digital video;

a processor configured to;

generate motion vectors corresponding to the motion of objects from a first frame of the digital video to a second frame of the digital video;

associate a confidence level with each vector; and

estimate global motion between the first and seconds frames based on the vectors and the confidence level associated with each vector.

34. The processor system of claim 33, wherein the processor is configured to estimate global motion by:

assigning a weight to each motion vector based on the location of the motion vector within the first or second frame;

determining a weighted distance for each motion vector; and

estimating a global motion vector based on the weight and weighted distance associated with each motion vector.

35. The processor system of claim 33, wherein the processor is further configured to validate the global motion by comparing a sum of the confidence levels to a threshold.