Real-time image stabilization

• US 20060061661A1
• Filed: 08/22/2005
• Published: 03/23/2006
• Est. Priority Date: 08/23/2004
• Status: Active Grant

• Current Assignee: Intergraph Software Technologies Company
• Original Assignee: Intergraph Software Technologies Company
• Inventors: Whitaker, Sheila G., Grindstaff, Gene A.
• Granted Patent: US 7,859,569 B2
• Time in Patent Office: Days
• US Class Current: 348/208.990
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/66 of image moments or centre of gravity
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken

Abstract

An apparatus and method for stabilizing image frames in a video data stream. A weighted average or centroid of the intensity or hue associated with pixels vs. the horizontal and vertical position of each pixel is calculated for a reference frame in the video data stream. A corresponding centroid is calculated for a subsequent frame in the stream. This image frame is then translated so that the centroid of the subsequent frame and the centroid of the reference frame coincide, reducing artifacts from shaking of the video capture device. Alternatively, the video stream frames may be divided into tiles and centroids calculated for each tile. The centroids of the tiles of a subsequent frame are curve fit to the centroids of tiles in a reference frame. An affine transform is then performed on the subsequent frame to reduce artifacts in the image from movements of the video capture device.

Claims

1. A method for stabilizing an image in a video data stream comprising:

a. calculating a centroid of the image, the centroid consisting of a weighted average of a value associated with each pixel as a function of the pixel'"'"'s position in the image for at least a portion of the image; and

b. translating the image such that the centroid of the image and the centroid of the same portion of a reference image approximately coincide.

2. A method according to claim 1, wherein the value associated with each pixel is one of a hue and an intensity.

3. A method according to claim 1, wherein the value associated with each pixel is selected from a portion of a range of values associated with each pixel.

4. A method according to claim 1 further including:

c. updating the reference image to a subsequent image in the video data stream.

5. A method according to claim 4 wherein the distance the image has been translated determines updating the reference image.

6. A method according to claim 1, wherein the portion of the image is approximately equal to the entire image frame.

7. A method for stabilizing an image in a video data stream comprising:

a. dividing the image into at least two regions;

b. calculating a centroid of each region of the image, the centroid consisting of a weighted average of a value associated with each pixel of the region as a function of the pixel'"'"'s position in the region; and

c. transforming the image so that the centroids of the image and the centroids of a reference image approximately coincide.

8. A method according to claim 7, wherein transforming the image includes calculating a curve that best approximates the positions of the centroids of the image region with the positions of the centroids of the reference image.

9. A method according to claim 8 further including:

d. updating the reference image to a subsequent image in the video data stream.

10. A computer program product for use on a computer system for stabilizing an image in a video data stream, the computer program product comprising a computer usable medium having computer readable program code thereon, the computer readable program code including:

a. program code for calculating a centroid of the image, the centroid consisting of a weighted average of a value associated with each pixel as a function of the pixel'"'"'s position in the image for at least a portion of the image; and

b. program code for translating the image such that the centroid of the image and the centroid of the same portion of a reference image approximately coincide.

11. A computer program product according to claim 10, wherein the value associated with each pixel is one of a hue and an intensity.

12. A computer program product according to claim 10, wherein the value associated with each pixel is selected from a portion of a range of values associated with each pixel.

13. A computer program product according to claim 10, further including:

c. program code for updating the reference image to a subsequent image in the video data stream.

14. A computer program product according to claim 10, wherein the distance the image has been translated determines updating the reference image.

15. A computer program product according to claim 10, wherein the portion of the image is approximately equal to the entire image frame.

17. A computer program product according to claim 10, wherein program code for transforming the image includes program code for calculating a curve that best approximates the positions of the centroids of the image region with the positions of the centroids of the reference image.

18. A computer program product according to claim 10, further including:

d. program code for updating the reference image to a subsequent image in the video data stream.

16. A computer program product for use on a computer system for stabilizing an image in a video data stream, the computer program product comprising a computer usable medium having computer readable program code thereon, the computer readable program code including:

a. program code for dividing the image into at least two regions;

b. program code for calculating a centroid of each region of the image, the centroid consisting of a weighted average of a value associated with each pixel of the region as a function of the pixel'"'"'s position in the region; and

c. program code for transforming the image so that the centroids of the image and the centroids of a reference image approximately coincide.