Method, device and computer program product for image stabilization using color matching

• US 20030048359A1
• Filed: 09/06/2002
• Published: 03/13/2003
• Est. Priority Date: 09/07/2001
• Status: Active Grant

• Current Assignee: Intergraph Software Technologies Company
• Original Assignee: Intergraph Hardware Technologies Company (Hexagon AB)
• Inventors: Grindstaff, Gene Arthur, McKay, Therman Ward III, Fletcher, Susan Heath Calvin
• Granted Patent: US 6,654,049 B2
• Time in Patent Office: Days
• US Class Current: 348/208.6
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Abstract

A method for image stabilization of at least two digital images is disclosed. In such an embodiment, a first digital image having a plurality of pixels and at least a second image having a plurality of pixels are provided. Each pixel has an associated address for display and is representative of a color. Either a user of the system provides a color match range or the system provides a predetermined color matched range. A pixel is selected within the first digital image. Preferably the pixel is one that represents an item within the image that is either blurry due to movement of the item or appears jittery due to camera movement. Due to the vast number of available colors a pixel selected in a first image can be matched to a pixel in a second image within a range. The range allows for compensation due to lighting changes. Once the pixel in the first image is selected it can be compared to all pixels within the second image. Each pixel within the second image that is within the color match range is saved and the closest pixel color to the pixel from the first image is selected. The address of the pixels in the second image are then readdressed such that the address of the pixel located in the second image that has the closest color to the pixel in the first image now has the same display address as that of the pixel in the first image. The repositioned second digital image is then stored to memory.

Claims

1. A method for image stabilization of a first digital image having a plurality of pixels and at least a second image having a plurality of pixels wherein each pixel has an associated address for display and is representative of a color, the method comprising:

providing a color match range;

selecting a pixel within the first digital image;

identifying the color associated with the selected pixel in the first digital image;

locating a pixel in the second image that is within the color match range of the color associated with the pixel from the first digital image;

repositioning the second image, so that the address of the pixel in the second digital image is positioned at the address of the selected pixel in the first digital image; and

storing the repositioned second digital image to memory.

2. The method according to claim 1 further comprising:

selecting an image area to be used for locating a pixel in the second image.

3. The method according to claim 1, further comprising:

providing an alignment point for aligning the first digital image and the repositioned second digital image such that the pixel of the first frame is aligned with the alignment point and the pixel in the second digital image is aligned with the alignment point when displayed on a display device.

4. The method according to claim 1, further comprising displaying the first and the repositioned second digital image sequentially on a display device.

10. The method according to claim 1, wherein the color match range requires an identical color match.

12. The method according to claim 1 wherein a color space having millions of colors is used to represent the pixel colors.

14. The method according to claim 1, wherein if a pixel is not located within the color match range, applying a predetermined formula for repositioning the pixels of the second frame.

16. The method according to claim 1, further comprising:

receiving a signal containing information representative of a color match range.

5. A method for image stabilization of a digital sequence of images, wherein each image has a plurality of pixels and wherein each pixel has an associated address for display and is representative of a color, the method comprising:

selecting a beginning digital image and an ending digital image from the sequence creating a subset sequence;

selecting a pixel in the beginning digital image and a pixel in the ending digital image;

linearly interpolating between the color of the pixel in the beginning digital image and the color of the pixel in the ending digital image such that an interpolated color is determined for each image between the beginning digital image and the ending digital image;

for each interpolated color associated with an image, searching, within that image, for a pixel having the interpolated color within a predetermined range of colors;

repositioning each image between the beginning digital image and the ending digital image, so that the address of the pixel having the interpolated color within an image is positioned to the address of the pixel selected in the beginning image.

6. The method according to claim 5 further comprising displaying at least the subset sequence on a display.

7. The method according to claim 5 further comprising:

selecting an image area to be used for locating a pixel in the second image.

8. The method according to claim 5, further comprising:

providing an alignment point for aligning the first digital image and the repositioned second digital image such that the pixel of the first frame is aligned with the alignment point and the pixel in the second digital image is aligned with the alignment point when displayed on a display device.

9. The method according to claim 5, wherein the subset sequence includes all of the images within the sequence.

11. The method according to claim 5, wherein the predetermined range requires an identical color match.

13. The method according to claim 5 wherein the pixel colors are represented in a color space having millions of colors.

15. The method according to claim 5, wherein if a pixel is not located within the color match range, applying a predetermined formula for repositioning the pixels of the second frame.

17. The method according to claim 5, further comprising:

receiving a signal containing information representative of a color match range.

18. A method for image stabilization of a sequence of temporally displayed digital video images, wherein each image contains a number of pixels and each pixel has an associated display position, the method comprising:

receiving a signal containing information representative of a color associated with a pixel contained within a first image of the sequence of digital video images;

identifying a pixel within a second image having a color value within a threshold range from the color contained within the first image, wherein the threshold may be any value including zero; and

repositioning the second image such that the pixel within the second image having a color value within the threshold range resides at the same position as the pixel from the first image when temporally displayed.

19. The method according to claim 18, further comprising:

receiving a signal containing information representative of a color match range.

20. A method for image stabilization of a sequence of temporally displayed digital video images, wherein each image contains a number of pixels and each pixel has an associated display position, the method comprising:

receiving a signal containing information representative of a color associated with a pixel contained within a first image of the sequence of digital video images;

receiving a signal representative of a color contained in a third digital image;

interpolating between the color in the first digital video image and the color in the third digital video image to determine a searchable color in a second digital video image;

identifying a pixel within the second digital video image having a color distance within a threshold range from the searchable color, wherein the threshold may be any value including zero; and

repositioning the second image such that the pixel within the second image having a color distance within the threshold range resides at the same position as the pixel from the first image when displayed on a display device.

21. A system for image stabilization of a video sequence having a plurality of video frames, the system comprising:

a user interface; and

a processor;

wherein the processor includes a user interface module for receiving signals from the user interface;

wherein the signals include a signal representative of a pixel selection in a first video frame;

the processor also includes a location module for locating a pixel within a second frame having an associated color that is within a color range of a color associated with the pixel selection from the first video frame;

a repositioning module for readdressing the second video frame so that the located pixel of the second frame has an address that is the same address as the pixel selection form the first video frame; and

a storage module for storing the readdressed second video frame.

22. The system according to claim 21, further comprising:

memory.

23. A computer program product having a computer program thereon containing computer code for image stabilization of a first digital image having a plurality of pixels and at least a second image having a plurality of pixels wherein each pixel has an associated address for display and is representative of a color, the computer code comprising:

computer code for providing a color match range;

computer code for selecting a pixel within the first digital image;

computer code for identifying the color associated with the selected pixel in the first digital image;

computer code for locating a pixel in the second image that is within the color match range of the color associated with the pixel from the first digital image;

computer code for repositioning the second image, so that the address of the pixel in the second digital image is positioned at the address of the selected pixel in the first digital image; and

computer code for storing the repositioned second digital image to memory.

24. The computer program product according to claim 23 further comprising:

computer code for allowing user selection of an image area to be used for locating a pixel in the second image.

25. The computer program product according to claim 23, further comprising:

computer code for allowing a user to provide an alignment point for aligning the first digital image and the repositioned second digital image such that the pixel of the first frame is aligned with the alignment point and the pixel in the second digital image is aligned with the alignment point when displayed on a display device.

26. The computer program product according to claim 23, further comprising:

computer code for displaying the first and the repositioned second digital image sequentially on a display device.

33. The method according to claim 23, wherein the predetermined range requires an identical color match.

35. The computer program product according to claim 23 wherein the pixel colors are represented in a color space having millions of colors.

37. The computer program product according to claim 23, further comprising computer code for applying a predetermined formula for repositioning the pixels of the second frame if a pixel is not located within the color match range.

38. The computer program product according to claim 23, further comprising:

computer code for receiving a signal containing information representative of a color match range.

27. A computer program product for use with a computer having computer readable code thereon for image stabilization of a digital sequence of images, wherein each image has a plurality of pixels and wherein each pixel has an associated address for display and is representative of a color, the computer readable code comprising:

computer code for allowing user selection of a beginning digital image and an ending digital image from the sequence creating a subset sequence;

computer code for allowing user selection of a pixel in the beginning digital image and a pixel in the ending digital image;

computer code for interpolating between the color of the pixel in the beginning digital image and the color of the pixel in the ending digital image such that an interpolated color is determined for each image between the beginning digital image and the ending digital image;

computer code for searching, within that image, for a pixel having the interpolated color within a predetermined range of colors for each interpolated color associated with an image;

computer code for repositioning each image between the beginning digital image and the ending digital image, so that the address of the pixel having the interpolated color within an image is positioned to the address of the pixel selected in the beginning image.

28. The computer program product according to claim 27 further comprising:

computer code for displaying at least the subset sequence on a display.

29. The computer program product according to claim 27 further comprising:

computer code for allowing a user to select an image area to be used for locating a pixel in the second image.

30. The computer program product according to claim 27, further comprising:

computer code for allowing user selection of an alignment point for aligning the first digital image and the repositioned second digital image such that the pixel of the first frame is aligned with the alignment point and the pixel in the second digital image is aligned with the alignment point when displayed on a display device.

31. The computer program product according to claim 27, wherein the subset sequence includes all of the images within the sequence.

32. The computer program product according to claim 27, wherein the color match range requires an identical color match.

34. The computer program product according to claim 27 wherein a color space having millions of colors is used to represent the pixel colors.

36. The computer program product according to claim 27, further comprising computer code for applying a predetermined formula for repositioning the pixels of the second frame if a pixel is not located within the color match range.

39. The computer program product according to claim 27, further comprising:

computer code for receiving a signal containing information representative of a color match range.

40. A computer program product for use with a computer wherein the computer program product has computer readable code thereon for image stabilization of a sequence of temporally displayed digital video images, wherein each image contains a number of pixels and each pixel has an associated display position, the computer code comprising:

computer code for receiving a signal containing information representative of a color associated with a pixel contained within a first image of the sequence of digital video images;

computer code for identifying a pixel within a second image having a color value within a threshold range from the color contained within the first image, wherein the threshold may be any value including zero; and

computer code for repositioning the second image such that the pixel within the second image having a color value within the threshold range resides at the same position as the pixel from the first image.

41. The computer program product according to claim 40, further comprising:

computer code for receiving a signal containing information representative of a color match range.

42. A computer program product for use with a computer, the computer program product having computer readable code thereon for image stabilization of a sequence of temporally displayed digital video images, wherein each image contains a number of pixels and each pixel has an associated display position, the computer code comprising:

computer code for receiving a signal containing information representative of a color associated with a pixel contained within a first image of the sequence of digital video images;

computer code for receiving a signal representative of a color contained in a third digital image;

computer code for interpolating between the color in the first digital video image and the color in the third digital video image to determine a searchable color in a second digital video image;

computer code for identifying a pixel within the second digital video image having a color distance within a threshold range from the searchable color, wherein the threshold may be any value including zero; and

computer code for repositioning the second image such that the pixel within the second image having a color value within the threshold range resides at the same position as the pixel from the first image when displayed on a display device.