Method to Stabilize Digital Video Motion

• US 20080170125A1
• Filed: 03/24/2008
• Published: 07/17/2008
• Est. Priority Date: 01/18/2005
• Status: Active Grant

• Current Assignee: Primax Electronics Limited
• Original Assignee: Primax Electronics Limited
• Inventors: Jheng, Fu-Ming, Yang, Shih-Hsuan
• Granted Patent: US 7,768,551 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation

Abstract

A method for calculating an estimated hand motion vector of a plurality of images includes selecting a current video frame and a previous video frame from a video stream, dividing the current video frame into a plurality of macroblocks, selecting a subset of macroblocks from the current video frame, comparing the subset of macroblocks to the previous video frame within a two dimensional comparison range to generate a set of motion vectors, calculating a frame motion vector for the current video frame, estimating a steady motion vector for the current video frame by a first-order autoregression, and calculating a hand motion vector for the current video frame by taking a difference between the steady motion vector and the frame motion vector of the current video frame.

Claims

1. A method for calculating an estimated hand motion vector of a plurality of images, comprising:

selecting a current video frame and a previous video frame from a video stream;

dividing the current video frame into a plurality of macroblocks;

selecting a subset of macroblocks from the current video frame;

comparing the subset of macroblocks to the previous video frame within a two dimensional comparison range to generate a set of motion vectors;

calculating a frame motion vector for the current video frame;

estimating a steady motion vector for the current video frame by a first-order autoregression; and

calculating a hand motion vector for the current video frame by taking a difference between the steady motion vector and the frame motion vector of the current video frame.

2. The method of claim 1 wherein the frame motion vector of the current video frame is calculated by:

selecting a median horizontal component from the set of motion vectors; and

selecting a median vertical component from the set of motion vectors; and

forming the frame motion vector from the median horizontal component and the median vertical component.

3. The method of claim 1 further comprising assigning the frame motion vector for an initial frame of the video stream to be a first default vector, and assigning the steady motion vector of the initial frame of the video stream to be a second default vector.

4. The method of claim 3 where the first default vector is the null vector <

0,0>

.

5. The method of claim 3 where the second default vector is the null vector <

0,0>

.

6. The method of claim 3 where the second default vector is the first default vector.

7. The method of claim 1 further comprising estimating the steady motion vector for the current video frame by summing a product of a first variable and the steady motion vector for the previous video frame with a product of a second variable and the frame motion vector of the current video frame, wherein the sum of the first variable and the second variable equals a first constant.

8. The method of claim 7 where the first constant is 1.

9. The method of claim 7 further comprising calculating an estimated hand motion vector by subtracting the steady motion vector of the previous video frame from the frame motion vector of the current video frame.

10. The method of claim 9 further comprising setting the second variable to a minimum of a smoothing value or a correlation value.

11. The method of claim 10 where the smoothing value is about 0.4.

12. The method of claim 10 further comprising calculating the correlation value by:

taking a first square root of an inner product of the estimated hand motion vector of the current video frame and the estimated hand motion vector of the current video frame;

taking a second square root of an inner product of the steady motion vector of the previous video frame and the steady motion vector of the previous video frame;

calculating a dividend value by multiplying the first square root by the second square root;

calculating a divisor value of an inner product of the estimated hand motion vector of the current video frame and the steady motion vector of the previous video frame; and

calculating the correlation value by taking an absolute value of the divisor value divided by the dividend value.

13. The method of claim 12 further comprising:

setting the smoothing value to a first smoothing constant when a first absolute value of a horizontal component of the frame motion vector of the current video frame minus a horizontal component of the steady motion vector of the previous frame is greater than a first threshold value, and a second absolute value of a horizontal component of the frame motion vector of the previous video frame minus a horizontal component of the steady motion vector of the previous video frame of the previous video frame is greater than a second threshold value, and a third absolute value of a horizontal component of the frame motion vector of the previous video frame of the previous video frame minus a horizontal component of the steady motion vector of the previous video frame of the previous video frame of the previous video frame is greater than a third threshold value;

otherwise setting the smoothing value to a second smoothing constant.

14. The method of claim 13 where the first smoothing constant is 0.8 and the second smoothing constant is 0.1.

15. The method of claim 12 further comprising:

setting the smoothing value to a first smoothing constant when a first absolute value of a vertical component of the frame motion vector of the current video frame minus a vertical component of the steady motion vector of the previous frame is greater than a first threshold value, and a second absolute value of a vertical component of the frame motion vector of the previous video frame minus a vertical component of the steady motion vector of the previous video frame of the previous video frame is greater than a second threshold value, and a third absolute value of a vertical component of the frame motion vector of the previous video frame of the previous video frame minus a vertical component of the steady motion vector of the previous video frame of the previous video frame of the previous video frame is greater than a third threshold value;

otherwise setting the smoothing value to a second smoothing constant.

16. The method of claim 15 where the first smoothing constant is 0.8 and the second smoothing constant is 0.1.

17. The method of claim 1 further comprising:

dividing a horizontal magnitude of the hand motion vector by 2 when the horizontal magnitude of the accumulated motion vector is larger than a quantity generated by multiplying a horizontal range limit by a scaling factor; and

dividing the vertical magnitude of the hand motion vector by 2 when the vertical magnitude of the accumulated motion vector is larger than a quantity generated by multiplying a vertical range limit by a scaling factor.

18. The method of claim 1 further comprising estimating the steady motion vector for the current video frame by:

setting the horizontal component of the steady motion vector by summing a product of a first variable and a horizontal component of the steady motion vector for the previous video frame with a product of a second variable and a horizontal component of the frame motion vector of the current video frame, wherein the sum of the first variable and the second variable equals a first constant; and

setting the vertical component of the steady motion vector by summing a product of a third variable and a vertical component of the steady motion vector for the previous video frame with a product of a fourth variable and a vertical component of the frame motion vector of the current video frame, wherein the sum of the third variable and the fourth variable equals a second constant.

19. The method of claim 18 where the first constant is 1.

20. The method of claim 18 where the second constant is 1.

21. The method of claim 18 further comprising calculating an estimated hand motion vector by subtracting the steady motion vector of the previous video frame from the frame motion vector of the current video frame.

22. The method of claim 21 further comprising setting the second variable to a minimum of a horizontal smoothing value or a horizontal correlation value.

23. The method of claim 22 further comprising calculating the horizontal correlation value by:

taking a first square root of an inner product of the estimated hand motion vector of the current video frame and the estimated hand motion vector of the current video frame;

taking a second square root of an inner product of the steady motion vector of the previous video frame and the steady motion vector of the previous video frame;

calculating a dividend value by multiplying the first square root by the second square root;

calculating a divisor value of an inner product of the estimated hand motion vector of the current video frame and the steady motion vector of the previous video frame; and

calculating the horizontal correlation value by taking an absolute value of the divisor value divided by the dividend value.

24. The method of claim 22 where the horizontal smoothing value is about 0.4.

25. The method of claim 21 further comprising setting the fourth variable to a minimum of a vertical smoothing value or a vertical correlation value.

26. The method of claim 25 further comprising calculating the vertical correlation value by:

taking a first square root of an inner product of the estimated hand motion vector of the current video frame and the estimated hand motion vector of the current video frame;

taking a second square root of an inner product of the steady motion vector of the previous video frame and the steady motion vector of the previous video frame;

calculating a dividend value by multiplying the first square root by the second square root;

calculating a divisor value of an inner product of the estimated hand motion vector of the current video frame and the steady motion vector of the previous video frame; and

calculating the vertical correlation value by taking an absolute value of the divisor value divided by the dividend value.

27. The method of claim 26 where the vertical smoothing value is about 0.4.

28. The method of claim 26 further comprising:

setting the smoothing value to a first smoothing constant when a first absolute value of a horizontal component of the frame motion vector of the current video frame minus a horizontal component of the steady motion vector of the previous frame is greater than a first threshold value, and a second absolute value of a horizontal component of the frame motion vector of the previous video frame minus a horizontal component of the steady motion vector of the previous video frame of the previous video frame is greater than a second threshold value, and a third absolute value of a horizontal component of the frame motion vector of the previous video frame of the previous video frame minus a horizontal component of the steady motion vector of the previous video frame of the previous video frame of the previous video frame is greater than a third threshold value;

otherwise setting the smoothing value to a second smoothing constant.

29. The method of claim 28 where the first smoothing constant is 0.8 and the second smoothing constant is 0.1.

30. The method of claim 28 where the first smoothing constant is 0.8 and the second smoothing constant is 0.1.

31. The method of claim 26 further comprising:

setting the smoothing value to a first smoothing constant when a first absolute value of a vertical component of the frame motion vector of the current video frame minus a vertical component of the steady motion vector of the previous frame is greater than a first threshold value, and a second absolute value of a vertical component of the frame motion vector of the previous video frame minus a vertical component of the steady motion vector of the previous video frame of the previous video frame is greater than a second threshold value, and a third absolute value of a vertical component of the frame motion vector of the previous video frame of the previous video frame minus a vertical component of the steady motion vector of the previous video frame of the previous video frame of the previous video frame is greater than a third threshold value;

otherwise setting the smoothing value to a second smoothing constant.

32. A digital video recording device comprising a lens, a means of capturing a sequence of images, a storage device, and a processor that performs the method of claim 1.