Image stabilizer

• US 8,120,661 B2
• Filed: 07/26/2007
• Issued: 02/21/2012
• Est. Priority Date: 07/26/2006
• Status: Expired due to Fees

• Current Assignee: Human Monitoring Ltd.
• Original Assignee: Human Monitoring Ltd.
• Inventors: Rabinowitz, Nitzan, Dvir, Ira
• Primary Examiner(s): GILES, NICHOLAS G
• Application Number:
  US12/375,023
• Publication Number:
  US 20090256918A1
• Time in Patent Office: 1,671 Days
• Field of Search: 348/208.99, 348/208.1, 348/208.3, 348/208.4, 348/208.6, 348/333.01, 348/333.11
• US Class Current: 348/208.6
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

A method for digital video image stabilization, the method including: estimating, from at least one portion of a frame, global frame displacement between an initial reference digital video frame and a current frame in a video sequence of frames; verifying, for the entire frame, the validity of the estimated frame displacement; and compensating for the estimated frame displacement by aligning at least one frame in the sequence with respect to the initial reference frame; wherein the step of aligning includes producing a corrected motion vector for the frame to be aligned and displacing the frame within the video frames sequence in accordance with the corrected motion vector.

Claims

1. A method for digital video image stabilization, the method comprising:

estimating, in a processor, from at least one portion of a frame, global frame displacement between an initial reference digital video frame and a current frame in a video sequence of frames;

verifying, for the entire frame, validity of said estimated frame displacement; and

compensating for said estimated frame displacement by aligning at least one frame in said sequence with respect to said initial reference frame;

wherein said step of aligning includes;

producing a corrected motion vector for said frame to be aligned; and

displacing said frame within the video frames sequence in accordance with said corrected motion vector;

wherein the step of estimating global frame displacement comprises;

in a full resolution space, calculating a best match of a centric portion of the current frame as a single block of pixels associated with said current frame and said reference frame;

wherein said step of calculating the best match of the centric portion comprises calculating an overall fit (L1 norm) by down sampling and then employing a non-linear programming global optimization procedure;

wherein said non-linear programming global optimization procedure is selected from the group including Powell method, conjugate gradient method, and Down-Hill Simplex.

2. The method according to claim 1, wherein said step of aligning includes:

for an initial preset number of frames of a video sequence (GOP);

a. assigning a zero displacement to a first and a last frame in the GOP; and

b. correcting normalized frame displacements of said GOP by projecting normalized frame displacements of said GOP on a line connecting two end points of the GOP with a displacement coordinate of a frame in said sequence having a maximum displacement.

3. A method, for digital video image stabilization, the method comprising:

estimating, in a processor, from at least one portion of a frame, global frame displacement between an initial reference digital video frame and a current frame in a video sequence of frames;

verifying, for the entire frame, validity of said estimated frame displacement; and

compensating for said estimated frame displacement by aligning at least one frame in said sequence with respect to said initial reference frame;

wherein said step of aligning includes;

producing a corrected motion vector for said frame to be aligned; and

displacing said frame within the video frames sequence in accordance with said corrected motion vector; and

for an initial preset number of frames of a video sequence (GOP);

a. assigning a zero displacement to a first and a last frame in the GOP; and

b. correcting normalized frame displacements of said GOP by projecting normalized frame displacements of said GOP on a line connecting two end points of the GOP with a displacement coordinate of a frame in said sequence having a maximum displacement.

4. The method according to claim 3, wherein said step of compensating includes:

calculating a mean of accumulative global motion displacements values associated with a stack of frames of size N, ending at said current frame,

calculating a deviation of frame displacement of the current frame, resulting from said global frame displacement estimation, from said calculated mean of accumulative displacements values; and

setting compensation coordinates equal to a negative value of said deviation.

5. The method according to claim 4, wherein said stack of N frames includes frames before and after the current frame in the frames sequence.

6. The method of claim 3, wherein said steps of assigning a zero displacement and correcting said GOP'"'"'s normalized frame displacements are carried out using an adaptive number of frames.

7. The method according to claim 6, wherein a length of said GOP is set dynamically as a function of maximum allowable displacement distance with respect to initial frame coordinates in the GOP, and frame coordinates of said first and last frames of the GOP are fixed to be zero.

8. The method according to claim 6, wherein a number of frames for each axis is set dynamically and independently as a function of maximum allowable number of frames from said reference frame and maximum allowable vertical distance with respect to the GOP first frame.

9. The method according to claim 3, further comprising applying a cropping procedure to eliminate flickering of frame margins.

10. The method according to claim 9, comprising utilizing full resolution of an optical sensor for said method for digital video image stabilization, and displaying only a part of the frame which is in line with a stabilized target resolution of the frame.

11. The method according to claim 3, further comprising displaying stabilized video frames on an oversized CCD\CMOS that displays only a cropped center of a recorded video image.