Method for image stabilization by adaptive filtering

• US 7,254,279 B2
• Filed: 10/17/2003
• Issued: 08/07/2007
• Est. Priority Date: 10/17/2003
• Status: Active Grant

• Current Assignee: HTC Corporation
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chen, Mei
• Primary Examiner(s): Bella; Matthew C.
• Assistant Examiner(s): CUNNINGHAM, GREGORY F
• Application Number:
  US10/687,681
• Publication Number:
  US 20050213840A1
• Time in Patent Office: 1,390 Days
• Field of Search: 348/155, 348/208.3, 348/208.4, 348/208.13, 348/208.99, 348/402.1, 348/404.1, 348/407.1, 348/412.1, 348/415.1, 348/416.1, 348/431.1, 348/451, 348/452, 348/628, 348/669, 348/700, 348/E7.007, 348/E7.013, 348/FOR.118, 348/123, 348/129, 348/131, 348/132, 348/151, 348/185, 382/236, 382/261, 396/55, 396/153, 396/421, 396/492, 396/FOR.889
• US Class Current: 382/261
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

A method for removing undesired motion from a sequential plurality of related images utilizes adaptive filtering of image motion to determine image shifts. Differences between sequential image motions are taken with respect to a reference frame and a smoothing filter is applied to such differences of a sample of images adjacent a test image. Image shift values are calculated by subtracting the difference associated with a test frame from a filtered sample that includes the test frame. The image shift values are compared to an image shift parameter and sample sizes adjusted and image shift values calculated in accordance with the adjusted sample sizes so that image stability is obtained without boundary flicker.

Claims

1. A method for removing undesired motion from a sequential plurality of related images, each of such images including a stationary component and being fixed within a frame of uniform dimension, said method comprising the steps of:

determining a plurality of sequential inter-frame movements of said stationary component over a plurality of frames, each of said inter-frame movements being associated with one of said plurality of frames;

choosing a reference frame, said reference frame having an associated inter-frame movement;

measuring the difference between each of said inter-frame movements and the inter-frame movement associated with said reference frame;

choosing an image shift parameter;

choosing a smoothing filter;

designating a test frame;

selecting at least one additional frame adjacent said test frame, said at least one additional frame and said test frame defining a sample;

applying said smoothing filter to the difference values of said sample to obtain a filtered difference value;

subtracting the difference associated with said test frame from said filtered value to obtain an image shift value;

comparing said image shift value to said image shift parameter;

adjusting the number of frames of said sample in accordance with said comparison;

calculating said image shift value on the basis of said adjusted number of frames of said sample;

shifting the position of the image associated with said test frame and image shift value with respect to said test frame by said image shift value;

designating the next frame of said sequence as said test frame.

2. A method as defined in claim 1 further including the step of enlarging all of said frames of said sequence by a zoom factor related to said image shift parameter.

3. A method as defined in claim 2 wherein the step of adjusting the number of frames of said sample in accordance with said calculated image shift value includes the steps of:

increasing the number of frames of said sample when said calculated image shift value is less than said image shift parameter;

then

calculating an adjusted image shift value in accordance with said increased number of frames;

continuing to increase said number of frames of said sample until said image shift value is no longer less than said image shift parameter;

decreasing the number of frames of said sample when said image shift value exceeds said image shift parameter.

4. A method as defined in claim 1 wherein the step of choosing a reference frame further includes the step of selecting the first frame of said sequential plurality of related images as said reference frame.

5. A method as defined in claim 1 wherein the step of selecting at least one additional frame further includes the step of selecting unequal numbers of frames prior to and after said test frame in said sequential plurality of related images.

6. A method as defined in claim 1 wherein the step of selecting at least one additional frame further includes the step of selecting equal numbers of frames prior to and after said test frame in said sequential plurality of related images.

7. A method as defined in claim 1 wherein the step of selecting a smoothing filter includes the step of selecting a Gaussian filter.

8. A method as defined in claim 1 wherein the step of selecting a smoothing filter includes the step of selecting a Kalman filter.

9. A method as defined in claim 1 wherein the step of selecting a smoothing filter includes the step of selecting a box filter.

10. A method for removing undesired motion from a sequential plurality of related images, each of such images including a stationary component and being fixed within a frame of uniform dimension, said method comprising the steps of:

determining a plurality of sequential inter-frame movements of said stationary component over said plurality of frames, each of said inter-frame movements being associated with one of said plurality of frames;

choosing a reference frame, said reference frame having an associated inter-frame movement;

measuring the difference between each of said inter-frame movements and the inter-frame movement associated with said reference frame;

choosing an image shift parameter;

choosing a smoothing filter;

designating a test frame;

selecting at least one additional frame adjacent said test frame, said at least one additional frame and said tesframe defining a sample;

applying said smoothing filter to difference values of said sample to obtain a filtered difference value;

subtracting the difference associated with said test frame from said filtered difference value to obtain an image shift value;

shifting the position of the image associated test frame and image shift value with respect to said test frame by said image shift value;

designating the next-frame of said sequence as said test frame;

continuing to designate test frames from said sequence until each frame of said sequence has been designated and the image fixed thereon shifted in accordance with said above-described process; and

then

enlarging all of said frames of said sequence by a zoom factor related to said image shift parameter.

11. A method as defined in claim 10 further including the steps of:

comparing said image shift value to said image shift parameter;

adjusting the number of frames of said sample in accordance with said comparison;

calculating said image shift value on the basis of said adjusted number of frames of said sample.

12. A method as defined in claim 11 wherein the step of adjusting the number of frames of said sample in accordance with said calculated image shift value includes the steps of:

increasing the number of frames of said sample when said calculated image shift value is less than said image shift parameter;

then

calculating an adjusted image shift value in accordance with said-increased number of frames;

continuing to increase said number of frames of said sample until said image shift value is no longer less than said image shift parameter;

decreasing the number of frames of said sample when said image shift value exceeds said image shift parameter.

13. A method as defined in claim 10 wherein the step of choosing a reference frame further includes the step of selecting the first frame of said sequential plurality of related images as said reference frame.

14. A method as defined in claim 10 wherein the step of selecting at least one additional frame further includes the step of selecting unequal numbers of frames prior to and after said test frame in said sequential plurality of related images.

15. A method as defined in claim 10 wherein the step of selecting at least one additional frame further includes the step of selecting equal numbers of frames prior to and after said test frame in said sequential plurality of related images.

16. A method as defined in claim 10 wherein the step of selecting a smoothing filter includes the step of selecting a Gaussian filter.

17. A method as defined in claim 10 wherein the step of selecting a smoothing filter includes the step of selecting a Kalman filter.

18. A method as defined in claim 10 wherein the step of selecting a smoothing filter includes the step of selecting a box filter.