Digital image stabilization method for correcting horizontal inclination distortion and vertical scaling distortion

• US 20080225127A1
• Filed: 03/11/2008
• Published: 09/18/2008
• Est. Priority Date: 03/12/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Ming, Zhao
• Granted Patent: US 7,999,856 B2
• Time in Patent Office: Days
• US Class Current: 348/208.99
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Provided is a digital image stabilization method. The method includes: extracting characterizing points from a current frame; matching the characterizing points of the current frame with characterizing points of a previous frame; detecting an outlier from the matched characterizing points and removing the outlier; calculating homography using the characterizing points from which the outlier has been removed; and correcting the current frame using the homography.

Claims

1. A digital image stabilization method for correcting camera shake of an image capture apparatus, the method comprising:

extracting characterizing points from a current frame;

matching the characterizing points of the current frame with characterizing points of a previous frame;

detecting an outlier from the matched characterizing points and removing the outlier;

calculating homography using the characterizing points after the outlier has been removed; and

correcting the current frame using the homography.

2. The method of claim 1, wherein the correcting of the current frame comprises:

accumulating the homography and previously calculated homography;

determining whether each value of parameters constituting the accumulated homography is within a specific range; and

if each value is within the specific range, correcting the current frame by inversely transforming the current frame using the accumulated homography.

3. The method of claim 1, wherein the homography is a matrix in the form of the following,

$\left[\begin{array}{ccc}1& h_2& h_3\\ 0& h_5& h_6\\ 0& 0& 1\end{array}\right]\hspace{1em}$

wherein h₂ denotes an amount of shear change, h₅ denotes an amount of scaling change, and h₃ and h₆ denote amounts of location change.

4. The method of claim 1, wherein the extracting of the characterizing points comprises:

searching for a salient region of an image using a saliency map; and

extracting the characterizing points using luminescence of a pixel included in the salient region.

5. The method of claim 4, wherein a Kanade-Lucas-Tomasi corner detector (KLT-CD) algorithm is used to extract the characterizing points.

6. The method of claim 1, further comprising:

comparing an average luminescence of the current frame with that of the previous frame; and

if the average luminescence differs from each other, correcting the difference in the average luminescence, prior to the matching of characterizing points.

7. The method of claim 1, wherein a normalized cross correlation (NCC) algorithm is used to match the characterizing points.

8. The method of claim 1, wherein a random sample consensus (RANSAC) algorithm is used to detect the outlier.

9. The method of claim 1, wherein a Leverg-Marquardt (LM) non-linear minimization algorithm is used to calculate the homography.

10. The method of claim 1, wherein the method is applied to a CMOS image sensor.

11. A digital image stabilization method for correcting camera shake of an image capture apparatus, the method comprising:

extracting characterizing points from a current frame;

matching the characterizing points of the current frame with characterizing points of a previous frame;

detecting an outlier from the matched characterizing points;

determining whether the outlier is greater than a threshold value;

if the outlier is greater than the threshold value, calculating a motion vector indicating the distance and direction of movement of an image using an image projection method;

if the outlier is less than the threshold value, removing the outlier and calculating homography using the characterizing points after the outlier has been removed; and

correcting the current frame using the motion vector or the homography.

12. The method of claim 11, wherein the correcting of the current frame comprises:

accumulating the homography and previously calculated homography;

determining whether each value of parameters constituting the accumulated homography is within a specific range; and

if each value is within the specific range, correcting the current frame by inversely transforming the current frame using the accumulated homography.

13. The method of claim 11, wherein the homography is a matrix in the form of the following,

$\left[\begin{array}{ccc}1& h_2& h_3\\ 0& h_5& h_6\\ 0& 0& 1\end{array}\right]\hspace{1em}$

wherein h₂ denotes an amount of shear change, h₅ denotes an amount of scaling change, and h₃ and h₆ denote amounts of location change.

14. The method of claim 11, wherein the extracting of the characterizing points comprises:

searching for a salient region of an image using a saliency map; and

extracting the characterizing points using luminescence of a pixel included in the salient region.

15. The method of claim 14, wherein a KLT-CD algorithm is used to extract the characterizing point.

16. The method of claim 11, further comprising:

comparing an average luminescence of the current frame with that of the previous frame; and

if the average luminescence differs from each other, correcting the difference in the average luminescence, prior to the matching of characterizing points.

17. The method of claim 11, wherein an NCC algorithm is used to match the characterizing points.

18. The method of claim 11, wherein a RANSAC algorithm is used to detect the outlier.

19. The method of claim 11, wherein an LM non-linear minimization algorithm is used to calculate the homography.

20. The method of claim 11, wherein the method is applied to a CMOS image sensor.