Image stabilization

• US 8,385,732 B2
• Filed: 07/29/2011
• Issued: 02/26/2013
• Est. Priority Date: 07/29/2011
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zuniga, Oscar, Voss, Shane D, Yost, Jason
• Primary Examiner(s): Laballe, Clayton E
• Assistant Examiner(s): Reisner, Noam
• Application Number:
  US13/194,027
• Publication Number:
  US 20130027775A1
• Time in Patent Office: 578 Days
• Field of Search: 396/52, 396/153, 348/208.3
• US Class Current: 396/52
• CPC Class Codes: G02B 27/646 compensating for small deviations, e.g. due to vibration or shake movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68

Abstract

Example embodiments disclosed herein relate to image stabilization. Image stabilization includes techniques used to reduce jitter associated with the motion of a camera. Image stabilization can compensate for pan and tilt (angular movement, equivalent to yaw and pitch) of a camera or other imaging device. Image stabilization can be used in still and video cameras, including those found in mobile devices such as cell phones and personal digital assistants (PDAs).

Claims

1. A method for image stabilization, comprising:

determining an amount of motion occurring between an original frame and an adjacent frame in a sequence;

analyzing the determined amount of motion occurring between the original frame and the adjacent frame in the sequence;

determining an intended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence by utilizing filtered frame-to-frame motion parameters rather than filtered cumulative motion parameters;

determining an unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence by utilizing filtered frame-to-frame motion parameters rather than filtered cumulative motion parameters;

centering a sub-window on the original frame; and

rendering a new frame as a warped version of the sub-window based on the determined intended motion component and the determined unintended motion component to remove the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

2. The method for image stabilization of claim 1, further comprising cropping the sub-window centered on the original frame to address undefined areas caused by the rendering of the new frame as a warped version of the sub-window.

3. The method for image stabilization of claim 1, further comprising moving the sub-window in a direction opposite the determined unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

4. The method for image stabilization of claim 1, wherein determining the amount of motion occurring between an original frame and the adjacent frame in the sequence includes utilizing feature based image registration.

5. The method for image stabilization of claim 1, wherein the sequence includes a plurality of video frames.

6. The method for image stabilization of claim 1, wherein determining the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence includes calculating a difference between the determined amount of motion occurring between the original frame and the adjacent frame in the sequence and the determined intended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

7. The method for image stabilization of claim 1, wherein determining the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence includes creating and utilizing a first order kinematic model for Kalman filtering of cumulative affine motion parameters.

8. The method of image stabilization of claim 1, wherein rendering a new frame as a warped version of the sub-window to remove the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence includes performing an affine warping.

9. The method for image stabilization of claim 1, in a mobile device.

10. A non-transitory computer-readable storage medium storing instructions for image stabilization that, when executed by a processor, cause the processor to:

determine an amount of motion occurring between an original frame and an adjacent frame in a sequence;

analyze the determined amount of motion occurring between the original frame and the adjacent frame in the sequence;

determine an intended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence by utilizing filtered frame-to-frame motion parameters rather than filtered cumulative motion parameters;

determine an unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence by utilizing filtered frame-to-frame motion parameters rather than filtered cumulative motion parameters;

center a sub-window on the original frame; and

render a new frame as a warped version of the sub-window based on the determined intended motion component and the determine unintended motion component to remove the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

11. The non-transitory computer-readable storage medium of claim 10, further comprising storing instructions for image stabilization that, when executed by a processor, cause the processor to crop the sub-window centered on the original frame to address undefined areas caused by to the rendering of the new frame as a warped version of the sub-window.

12. The non-transitory computer-readable storage medium of claim 10, further comprising storing instructions for image stabilization that, when executed by a processor, cause the processor move the sub-window in a direction opposite the determined unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

13. The non-transitory computer-readable storage medium of claim 10, wherein feature based image registration is utilized by the processor to determine the amount of motion occurring between an original frame and the adjacent frame in the sequence.

14. The non-transitory computer-readable storage medium of claim 10, wherein the sequence includes a plurality of video frames.

15. The non-transitory computer-readable storage medium of claim 10, wherein the processor calculates a difference between the determined amount of motion occurring between the original frame and the adjacent frame in the sequence and the determined intended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence to determine the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

16. The non-transitory computer-readable storage medium of claim 10, wherein the processor creates and utilizes a first order kinematic model for Kalman filtering of cumulative affine motion parameters to determine the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

17. The non-transitory computer-readable storage medium of claim 10, wherein the processor performs an affine warping to render a new frame as a warped version of the sub-window to remove the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

18. The non-transitory computer-readable storage medium of claim 10, in a mobile device including the processor.

19. A mobile device having image stabilization, comprising:

a processor;

a camera; and

a non-transitory computer-readable storage medium storing instructions for image stabilization that are executed by the processor to determine an amount of motion occurring between an original frame and an adjacent frame in a sequence, analyze the determined amount of motion occurring between the original frame and the adjacent frame in the sequence, determine an intended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence by utilizing filtered frame-to-frame motion parameters rather than filtered cumulative motion parameters, determine an unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence by utilizing filtered frame-to-frame motion parameters rather than filtered cumulative motion parameters, center a sub-window on the or frame, and render a new frame as a warped version of the sub-window based on the determined intended motion component and the determined unintended motion component to remove the unintended motion component of the determined amount of motion occurring between the original frame and the adjacent frame in the sequence.

20. The mobile device of claim 19, wherein the sequence includes a plurality of video frames.