IMAGE STABILIZER

• US 20090256918A1
• Filed: 07/26/2007
• Published: 10/15/2009
• Est. Priority Date: 07/26/2006
• Status: Active Grant

• Current Assignee: Human Monitoring Ltd.
• Original Assignee: Human Monitoring Ltd.
• Inventors: Rabinowitz, Nitzan, Dvir, Ira
• Granted Patent: US 8,120,661 B2
• Time in Patent Office: Days
• US Class Current: 348/208.400
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

A method for digital video image stabilization, the method including: estimating, from at least one portion of a frame, global frame displacement between an initial reference digital video frame and a current frame in a video sequence of frames; verifying, for the entire frame, the validity of the estimated frame displacement; and compensating for the estimated frame displacement by aligning at least one frame in the sequence with respect to the initial reference frame; wherein the step of aligning includes producing a corrected motion vector for the frame to be aligned and displacing the frame within the video frames sequence in accordance with the corrected motion vector.

Claims

1. A method for digital video image stabilization, the method comprising:

estimating, from at least one portion of a frame, global frame displacement between an initial reference digital video frame and a current frame in a video sequence of frames;

verifying, for the entire frame, validity of said estimated frame displacement; and

compensating for said estimated frame displacement by aligning at least one frame in said sequence with respect to said initial reference frame;

wherein said step of aligning includes;

producing a corrected motion vector for said frame to be aligned; and

displacing said frame within the video frames sequence in accordance with said corrected motion vector.

2. The method according to claim 1, further including aligning a plurality of frames in said video frames sequence with respect to said initial reference frame.

3. The method according to claim 2, wherein said step of compensating includes:

calculating a mean of accumulative global motion displacements values associated with a stack of frames of size N, ending at a current frame,

calculating a deviation of frame displacement of the current frame, resulting from its global motion estimation, from said calculated mean accumulative displacements; and

setting compensation coordinates equal to a negative value of said deviation.

4. The method according to claim 3, wherein said stack of N frames includes frames before and after the current frame in the frames sequence.

14. The method according to claim 1, wherein the step of estimating global frame displacement comprises:

in a full resolution space, calculating a best match of a centric portion of the current frame as a single block of pixels associated with said current frame and said reference frame.

15. The method according to claim 14, wherein said step of calculating the best match of the centric portion comprises calculating an overall fit (L1 norm) by down sampling and then employing a non-linear programming global optimization procedure.

16. The method according to claim 15, wherein said non-linear programming global optimization procedure is selected from the group including Powell method, conjugate gradient method, and Down-Hill Simplex.

17. The method according to claim 1, wherein said step of aligning includes:

for an initial preset number of frames of a video sequence (GOP);

a. assigning a zero displacement to a first and a last frame in the GOP; and

b. projecting normalized frame displacements of said GOP on a line connecting two end points of the GOP with a displacement coordinate of a frame in said sequence having a maximum displacement, so as to correct said normalized frame displacement.

18. The method of claim 17, wherein said procedure to align frame displacements with respect to the initial reference frame is carried out using an adaptive number of frames.

19. The method according to claim 18, wherein said GOP length is set dynamically as a function of maximum allowable displacement distance with respect to initial frame coordinates in the GOP, and frame coordinates of said first and last frames of the GOP are fixed to be zero.

20. The method according to claim 18, wherein a number of frames for each axis is set dynamically and independently as a function of maximum allowable number of frames from said reference frame and maximum allowable vertical distance with respect to the GOP initial frame.

22. The method according to claim 1, further comprising applying a cropping procedure to eliminate flickering of frame margins.

23. The method according to claim 22, comprising utilizing full resolution of an optical sensor for the stabilization process, and displaying only a part of the frame which is in line with a stabilized target resolution of the frame.

24. The method according to claim 1, further comprising displaying stabilized video frames using an oversized CCD\CMOS that displays only a cropped center of a recorded video image.

25. The method according to claim 14, wherein said step of aligning includes:

for an initial preset number of frames of a video sequence (GOP);

a. assigning a zero displacement to a first and a last frame in the GOP; and

b. projecting normalized frame displacements of said GOP on a line connecting two end points of the GOP with a displacement coordinate of a frame in said sequence having a maximum displacement, so as to correct said normalized frame displacement.

26. The method according to claim 14, further comprising applying a cropping procedure to eliminate flickering of frame margins.

27. The method according to claim 17, further comprising applying a cropping procedure to eliminate flickering of frame margins.

28. The method according to claim 14, further comprising displaying stabilized video frames using an oversized CCD\CMOS that displays only a cropped center of a recorded video image.

29. The method according to claim 17, further comprising displaying stabilized video frames using an oversized CCD\CMOS that displays only a cropped center of a recorded video image.

30. The method according to claim 22, further comprising displaying stabilized video frames using an oversized CCD\CMOS that displays only a cropped center of a recorded video image.

5-13. -13. (canceled)

21. (canceled)