METHOD AND DEVICE FOR DIGITAL IMAGE STABILIZATION

• US 20110037861A1
• Filed: 08/09/2006
• Published: 02/17/2011
• Est. Priority Date: 08/10/2005
• Status: Active Grant

• Current Assignee: Goodix Technology Co., Ltd. (Shenzhen Goodix Technology Co., Ltd.)
• Original Assignee: NXP B.V. (NXP Semiconductors NV)
• Inventors: Auberger, Stephane, Miro, Carolina, Picard, Yann
• Granted Patent: US 8,363,115 B2
• Time in Patent Office: Days
• US Class Current: 348/208.4
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

The invention concerns a method for digital image stabilization for removing jitter from an original sequence of images (10) generated by a camera. The original sequence (10) is applied to a stabilization algorithm (11). The global motion of a camera is estimated and filtered (110) using a default motion filter. Predetermined parameters (13, 140, 141) are extracted from both the original (10) and stabilized (12) sequences of images. A measure value is computed in order to evaluate (15) the stabilization quality and compared to a threshold. Depending on the results of the evaluation (O₁₅, O′₁₅) the stabilization algorithm (11) uses an alternative filter, in order to improve stabilization quality, or continues to use the default filter.

Claims

1. A device for digital image stabilization for removing unwanted camera movements, called jitter, from an original sequence of images generated by said camera and obtaining a stabilized sequence of images, said original sequence being applied to a stabilization algorithm, in order to remove said jitter, said device comprising:

an initial stage with a first motion filter having a first set of predetermined characteristics, a default filter, and at least one alternative motion filter having a second set of predetermined characteristics, said filters being implemented in said stabilization algorithm;

a first stage in which a global motion of said camera and filtering said global motion with said default motion filter is estimated;

a second stage in which predetermined parameters from both said original and stabilized sequences are extracted and a measure value, called a mark, is generated, in order of evaluating stabilization quality; and

a third stage in which said mark is compared to a predetermined threshold and a first control signal is generated if said threshold is exceeded, otherwise a second control signal is generated, said first control signal forcing said stabilization algorithm to use an alternative filter, in order to improve stabilization quality, and said second control signal forcing stabilization algorithm to continue to use said default filter.

2. A method for digital image stabilization for removing unwanted camera movements, called jitter, from an original sequence of images generated by said camera and obtaining a stabilized sequence of images, said original sequence being applied to a stabilization algorithm, in order to remove said jitter, the method comprising:

an initial step of building a first motion filter having a first set of predetermined characteristics, a default filter, and at least one alternative motion filter having a second set of predetermined characteristics, said filters being implemented in said stabilization algorithm;

a first step of estimating a global motion of said camera and in filtering said global motion with said default motion filter,

a second step of extracting predetermined parameters from both said original and stabilized sequences and generating a measure value, said mark, in order of evaluating stabilization quality; and

a third step of comparing said mark to a predetermined threshold and generating a first control signal if said threshold is exceeded, otherwise generating a second control signal, said first control signal forcing said stabilization algorithm to use an alternative filter, in order to improve stabilization quality, and said second control signal forcing said stabilization algorithm to continue to use said default filter.

3. A method as claimed in claim 2, characterized in that, said images being divided in predetermined large areas of pixels, called “

macro blocks”

, said parameters comprise global motion vectors and “

macro blocks”

vectors.

4. A method as claimed in claim 3, characterized in that said mark comprises a stabilization quality metric based on a combination of measures made on both said original and stabilized sequences, said measures comprising at least a peak signal-to-noise ratio of consecutive images and a frequency analysis of said global motion vectors along a sequence of predetermined number of images, said measures being performed on both said original and stabilized sequences of images.

5. A method as claimed in claim 4, comprising a step comprising dividing the motion spectrum of said sequence of predetermined number of images in two predetermined energy bands, a first energy band of low frequencies and a second energy band of high frequencies, in that said motion spectrum is computed from consecutive global motion vectors, and in that said stabilization quality metric is based on the computation and the combination of first and second parameters, called A_below and A_above, respectively, representing the proportion of energy in said first and second energy band, respectively, of third and fourth parameters representing the reduction of motion energy between said original and stabilized sequences of images over said first and second frequency bands, and a fifth parameter, called inter-frame transformation fidelity, representing said peak signal-to-noise ratio between consecutive images of said sequence of predetermined number of images.

6. A method as claimed in claim 5, characterized in that said high frequencies are the frequencies above 1 Hz and said low frequencies are the frequencies under 1 Hz.

7. A method as claimed in claim 5, characterized in that said first set of characteristics of the default motion filter, called FILTER_C, is designed in order to obtain a high stabilization of the motion of said camera, and comprises a correspondence table filled with high damping factor value.

8. A method as claimed in claim 7, characterized in that

said second set of characteristics of at least one alternative filter, called FILTER₀, is designed in order to allow a faster tracking of large intentional motion of said camera, and comprises a correspondence table filled with damping factors whose value are lower than the values of the damping factors of said FILTER_C.

9. A method as claimed in claim 5, in which said initial step comprises the building of two further alternative motion filters having third and fourth set of characteristics, called FILTER_A and FILTER_B, respectively, in that said third set of characteristics are so designed that a moving average filtering is performed on said original sequence of images, said global moving vectors being replaced by an average of a predetermined number of previous global moving vectors, representing a sampling window, and high frequencies components of said vectors being filtered, and in that said fourth set of characteristics are so designed that a double-pass moving average filtering is performed on said original sequence of images, said global moving vectors being replaced by an average of a predetermined number of previous global moving vectors and high frequencies components of said vectors being filtered during a first pass, the so-filtered moving vectors being filtered again over the same sampling window, during a second pass.

10. A method as claimed in claim 9, in which, said initial step comprises the experimental determination of three further thresholds, called T\, T̂

and T₃, respectively, and in that, if said first threshold is exceeded, it comprises a further step of successive comparisons, using said first and second parameters, A_be\ow and Aabove, respectively;

If A_below>

T1*A_above then said filters FILTER_A or FILTER_B are used, and If A_bove>

T1, then said filter FILTER_B is used, otherwise said filter FILTER_A is used; and

If Abelow<

T\*A_above then the FILTER_C or FILTER_C filters are used, and If Aabove>

T1, then said FILTER_C is used, otherwise said FILTER₀ is used.

11. A method as claimed in claim 10, characterized in that said sampling window is a sliding window comprising a constant number of time slots, each time slot corresponding to an interval of time separating two successive frames of both said original and stabilized sequence, said sliding window shifting with the time from one frame to the next one, in order to perform said motion filtering in real time.