Automated stabilization method for digital image sequences

• US 6,809,758 B1
• Filed: 12/29/1999
• Issued: 10/26/2004
• Est. Priority Date: 12/29/1999
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Jones, Paul W.
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Jerabek, Kelly L.
• Application Number:
  US09/474,723
• Time in Patent Office: 1,763 Days
• Field of Search: 348/208, 348/208.99, 348/208.1, 348/208.4, 348/208.3, 348/208.12, 348/699, 348/700, 382/168, 382/172
• US Class Current: 348/208.99
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A method and structure for stabilizing a motion image formed using a sequence of successive frames which includes calculating a motion vector field between adjacent frames; forming a motion vector histogram from horizontal and vertical components of the motion vector field; applying a threshold to the motion vector histogram to produce a thresholded motion vector histogram; generating average horizontal and vertical motion components from the thresholded motion vector histogram; filtering the average horizontal and vertical motion components over a number of frames to identify unwanted horizontal and vertical motion components for each of the frames; and stabilizing the image sequence by shifting each frame according to the corresponding unwanted horizontal and vertical motion components.

Claims

1. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating motion vectors for each of said frames, wherein said motion vectors follow corresponding points between adjacent frames;

forming a motion vector histogram for each of said frames from horizontal and vertical components of said motion vectors;

removing unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

generating average horizontal and vertical components from said processed motion vector histogram for each of said frames;

filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames; and

translating each frame by said unwanted horizontal and vertical components.

2. The method in claim 1, wherein said filtering comprises highpass filtering.

3. The method in claim 1, wherein said removing includes discarding motion vectors from said motion vector histogram, which have values below a threshold.

4. The method in claim 3, wherein said threshold is determined adaptively for each of said frames.

5. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating motion vectors for each of said frames, wherein said motion vectors follow corresponding points between adjacent frames;

forming a motion vector histogram for each of said frames from horizontal and vertical components of said motion vectors;

removing unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

generating average horizontal and vertical components from said processed motion vector histogram for each of said frames;

filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames; and

translating each frame by said unwanted horizontal and vertical components;

wherein said filtering is user-adjustable.

6. The method in claim 5, wherein said removing includes discarding values from said motion vector histogram below a threshold.

7. The method in claim 6, wherein said threshold is user-adjustable.

8. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating motion vectors for each of said frames, wherein said motion vectors follow corresponding points between adjacent frames;

forming a motion vector histogram for each of said frames from horizontal and vertical components of said motion vectors;

removing unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

generating average horizontal and vertical components from said processed motion vector histogram for each of said frames;

filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames; and

translating each frame by said unwanted horizontal and vertical components;

wherein said removing includes discarding values from said motion vector histogram below a threshold and said threshold is determined adaptively for each of said frames.

9. A computerized digital image system for stabilizing an image sequence formed from a number of successive frames comprising:

a motion estimator calculating motion vectors for each of said frames, wherein said motion vectors follow corresponding points between adjacent frames;

a histogram generator forming a motion vector histogram for each of said frames from horizontal and vertical components of said motion vectors;

a processor removing unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

an averaging unit generating average horizontal and vertical components from said processed histogram for each of said frames;

a filter filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components of each of said frames; and

a stabilizing unit translating each frame by said unwanted horizontal and vertical components.

10. The computerized digital imaging system in claim 9, wherein said filter comprises a highpass filter.

11. The computerized digital imaging system in claim 9, wherein said processor discards values from said motion vector below a threshold.

12. The computerized digital imaging system in claim 11, wherein said processor includes a user interface for adjusting said threshold.

13. A computerized digital image system for stabilizing an image sequence formed from a number of successive frames comprising:

a motion estimator calculating motion vectors for each of said frames, wherein said motion vectors follow corresponding points between adjacent frames;

a histogram generator forming a motion vector histogram for each of said frames from horizontal and vertical components of said motion vectors;

a processor removing unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

an averaging unit generating average horizontal and vertical components from said processed histogram for each of said frames;

a filter filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components of each of said frames; and

a stabilizing unit translating each frame by said unwanted horizontal and vertical components;

wherein said filter includes a user interface for adjusting said filter.

14. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating motion vectors for each of said frames;

forming a motion vector histogram for each of said frames;

generating average horizontal and vertical components from said motion vector histogram for each of said frames;

identifying unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

translating each frame by said unwanted horizontal and vertical components; and

further comprising removing unreliable motion vectors from said motion vector histogram for each of said frames.

15. The method in claim 14, wherein said motion vectors follow corresponding points between adjacent frames.

16. The method in claim 14, wherein said motion vector histogram comprises horizontal and vertical components of said motion vectors.

17. The method in claim 14, wherein said identifying includes filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames.

18. The method in claim 14, wherein said removing includes discarding values from said motion vector histogram below a threshold.

19. The method in claim 18, wherein said threshold is user-adjustable.

20. The method in claim 18, wherein said threshold is determined adaptively for each of said frames.

21. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating motion vectors for each of said frames;

forming a motion vector histogram for each of said frames;

generating average horizontal and vertical components from said motion vector histogram for each of said frames;

identifying unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

translating each frame by said unwanted horizontal and vertical components;

wherein said identifying includes filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames; and

wherein said filtering comprises highpass filtering.

22. The method in claim 21, wherein said motion vectors follow corresponding points between adjacent frames.

23. The method in claim 21, wherein said motion vector histogram comprises horizontal and vertical components of said motion vectors.

24. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating motion vectors for each of said frames;

forming a motion vector histogram for each of said frames;

generating average horizontal and vertical components from said motion vector histogram for each of said frames;

identifying unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

translating each frame by said unwanted horizontal and vertical components;

wherein said identifying includes filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames; and

wherein said filtering is user-adjustable.

25. The method in claim 24, wherein said motion vectors follow corresponding points between adjacent frames.

26. The method in claim 24, wherein said motion vector histogram comprises horizontal and vertical components of said motion vectors.

27. A method of using a computer program for stabilizing an image sequence formed from a plurality of successive frames comprising:

using said computer program to calculate motion vectors for each of said frames;

using said computer program to form a motion vector histogram for each of said frames;

using said computer program to generate average horizontal and vertical components from said motion vector histogram for each of said frames;

using said computer program to identify unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

using said computer program to translate each frame by said unwanted horizontal and vertical components; and

further comprising using said computer program to remove unreliable motion vectors from said motion vector histogram for each of said frames.

28. The method in claim 27, wherein said motion vectors follow corresponding points between adjacent frames.

29. The method in claim 27, wherein said motion vector histogram comprises horizontal and vertical components of said motion vectors.

30. The method in claim 27, wherein said using said computer program to identify unwanted horizontal and vertical components for each of said frames includes using said computer program to filter said average horizontal and vertical components from two or more successive frames.

31. The method in claim 27, wherein said using said computer program to remove includes using said computer program to discard values from said motion vector histogram below a threshold.

32. The method in claim 31, wherein said threshold is user-adjustable.

33. A method of using a computer program for stabilizing an image sequence formed from a plurality of successive frames comprising:

using said computer program to calculate motion vectors for each of said frames;

using said computer program to form a motion vector histogram for each of said frames;

using said computer program to generate average horizontal and vertical components from said motion vector histogram for each of said frames;

using said computer program to identify unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

using said computer program to translate each frame by said unwanted horizontal and vertical components; and

wherein said using said computer program to identify unwanted horizontal and vertical components for each of said frames includes using said computer program to filter said average horizontal and vertical components from two or more successive frames; and

wherein said using said computer program to filter comprises using said computer program to highpass filter.

34. A method of using a computer program for stabilizing an image sequence formed from a plurality of successive frames comprising:

using said computer program to calculate motion vectors for each of said frames;

using said computer program to form a motion vector histogram for each of said frames;

using said computer program to generate average horizontal and vertical components from said motion vector histogram for each of said frames;

using said computer program to identify unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

using said computer program to translate each frame by said unwanted horizontal and vertical components; and

wherein said using said computer program to identify unwanted horizontal and vertical components for each of said frames includes using said computer program to filter said average horizontal and vertical components from two or more successive frames; and

wherein said using said computer program to filter is user-adjustable.

35. A method of using a computer program for stabilizing an image sequence formed from a plurality of successive frames comprising:

using said computer program to calculate motion vectors for each of said frames;

using said computer program to form a motion vector histogram for each of said frames;

using said computer program to remove unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

using said computer program to generate average horizontal and vertical components from said motion vector histogram for each of said frames;

using said computer program to identify unwanted horizontal and vertical components for each of said frames based on said average horizontal and vertical components; and

using said computer program to translate each frame by said unwanted horizontal and vertical components;

wherein said using said computer program to remove includes using said computer program to discard values from said motion vector histogram below a threshold and said threshold is determined adaptively by said computer program for each of said frames.

36. A method for stabilizing an image sequence formed from a plurality of successive frames comprising:

calculating a plurality of motion vectors for each of said frames, wherein said motion vectors follow corresponding points between adjacent frames;

forming a motion vector histogram for each of said frames from horizontal and vertical components of said motion vectors;

removing unreliable motion vectors from said motion vector histogram to form a processed motion vector histogram for each of said frames;

generating average horizontal and vertical components from said processed motion vector histogram for each of said frames;

filtering said average horizontal and vertical components from two or more successive frames to identify unwanted horizontal and vertical components for each of said frames; and

translating each frame by said unwanted horizontal and vertical components;

wherein said filtering is adjustable.

37. The method of claim 36, wherein said filtering is user-adjustable.