Determining orientation of images containing blue sky

• US 6,512,846 B1
• Filed: 11/29/1999
• Issued: 01/28/2003
• Est. Priority Date: 11/29/1999
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US09/450,366
• Time in Patent Office: 1,156 Days
• Field of Search: 382/162-167, 382/224-228, 382/293-97, 358/598-527
• US Class Current: 382/165
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method, image recognition system, computer program, etc., for determining image orientation. The invention classifies potential sky pixels in the image by color, identifies spatially contiguous regions of the potential sky pixels, identifies actual sky regions by eliminating ones of the spatially contiguous regions that have a texture above a predetermined texture threshold, computes desaturation gradients of the actual sky regions, classifies the image as one of portrait and landscape based on average absolute values of horizontal and vertical desaturation gradient of pixels within each of the actual sky regions, determines orientation of the image based on a polarity of the average horizontal and vertical desaturation gradients, and confirms that the actual sky regions are true sky regions by comparing the desaturation gradients with a predetermined desaturation gradient for sky.

Claims

1. A method of determining image orientation comprising:

classifying potential sky pixels in said image by color;

identifying spatially contiguous regions of said potential sky pixels;

identifying actual sky regions by eliminating ones of said spatially contiguous regions that have a texture above a predetermined texture threshold;

computing desaturation gradients of said actual sky regions;

classifying said image as one of portrait and landscape based on average absolute values of horizontal and vertical desaturation gradient of pixels within each of said actual sky regions;

determining orientation of said image based on a polarity of said average horizontal and vertical desaturation gradients; and

confirming that said actual sky regions are true sky regions by comparing said desaturation gradients with a predetermined desaturation gradient for sky.

2. The method in claim 1, further comprising maintaining only ones of said spatially contiguous regions as said actual sky regions which:

have substantially dissimilar average horizontal and vertical gradients;

have a color distribution which is consistent with a predetermined color distribution for sky; and

contact a border of said image.

3. The method in claim 1, wherein said classifying said image as one of portrait and landscape includes:

classifying said image as said portrait if an absolute value of said average horizontal desaturation gradient is greater than an absolute value of said average vertical desaturation gradient; and

classifying said image as said landscape if an absolute value of said average horizontal desaturation gradient is less than an absolute value of said average vertical desaturation gradient.

4. The method in claim 1, wherein said desaturation gradients comprise desaturation gradients for red, green and blue trace components of said image.

5. The method in claim 1, wherein said predetermined desaturation gradient for sky comprises, from horizon to zenith, a decrease in red and green light trace components and a substantially constant blue light trace component.

6. The method in claim 1, wherein said classifying potential sky pixels in said image by color comprises:

forming a belief map of sky color pixels;

computing an adaptive threshold of sky color; and

determining if said potential sky pixels exceed said threshold.

7. The method in claim 6, wherein said computing of said adaptive threshold comprises identifying a first valley in a belief histogram derived from said belief map.

8. The method in claim 7, wherein said belief map and said belief histogram are unique to said image.

9. A method of determining image orientation comprising:

classifying sky pixels in said image by color;

computing desaturation gradients of said sky pixels;

classifying said image as one of portrait and landscape based on average absolute values of horizontal and vertical desaturation gradient of said sky pixels; and

determining orientation of said image based on a polarity of said average horizontal and vertical desaturation gradients.

10. The method in claim 9, further comprising:

identifying spatially contiguous regions of said sky pixels; and

identifying actual sky regions by eliminating ones of said spatially contiguous regions that have a texture above a predetermined texture threshold.

11. The method in claim 10, further comprising confirming that said actual sky regions are true sky regions by comparing said desaturation gradients with a predetermined desaturation gradient for sky.

12. The method in claim 10, further comprising maintaining only ones of said spatially contiguous regions as said actual sky regions which:

have substantially dissimilar average horizontal and vertical desaturation gradients;

have a color distribution which is consistent with a predetermined color distribution for sky; and

contact a border of said image.

13. The method in claim 9, wherein said classifying said image as one of portrait and landscape includes:

classifying said image as said portrait if an absolute value of said average horizontal desaturation gradient is greater than an absolute value of said average vertical desaturation gradient; and

classifying said image as said landscape if an absolute value of said average horizontal desaturation gradient is less than an absolute value of said average vertical desaturation gradient.

14. The method in claim 9, wherein said desaturation gradients comprise desaturation gradients for red, green and blue trace components of said image.

15. The method in claim 11, wherein said predetermined desaturation gradient for sky comprises, from horizon to zenith, a decrease in red and green light trace components and a substantially constant blue light trace component.

16. The method in claim 9, wherein said classifying sky pixels in said image by color comprises:

forming a belief map of sky color pixels;

computing an adaptive threshold of sky color; and

determining if said sky color pixels exceed said threshold.

17. The method in claim 16, wherein said computing of said adaptive threshold comprises identifying a first valley in a belief histogram derived from on said belief map.

18. The method in claim 17, wherein said belief histogram and said belief map are unique to said image.

19. A image orientation identification system comprising:

a color classifier classifying sky pixels in said image by color;

a logic unit computing desaturation gradients of said sky pixels;

a portrait/landscape classifier classifying said image as one of portrait and landscape based on average absolute values of horizontal and vertical desaturation gradient of said sky pixels; and

a comparator determining orientation of said image based on a polarity of said average horizontal and vertical desaturation gradients.

20. The image orientation identification system in claim 19, further comprising:

a second comparator identifying spatially contiguous regions of said sky pixels; and

a third comparator identifying actual sky regions by eliminating ones of said spatially contiguous regions that have a texture above a predetermined texture threshold.

21. The image orientation identification system in claim 20, further comprising a fourth comparator confirming that said actual sky regions are true sky regions by comparing desaturation gradients of said actual sky regions with a predetermined desaturation gradient for sky.

22. The image orientation identification system in claim 20, wherein said third comparator only maintains ones of said spatially contiguous regions as said actual sky regions that:

have substantially dissimilar average horizontal and vertical desaturation gradients;

have a color distribution which is consistent with a predetermined color distribution for sky; and

contact a border of said image.

23. The image orientation identification system in claim 19, wherein said portrait/landscape classifier:

classifies said image as said portrait if an absolute value of said average horizontal desaturation gradient is greater than an absolute value of said average vertical desaturation gradient; and

classifies said image as said landscape if an absolute value of said average horizontal desaturation gradient is less than an absolute value of said average vertical desaturation gradient.

24. The image orientation identification system in claim 21, wherein said desaturation gradients of said actual sky regions comprise desaturation gradients for red, green and blue trace components of said image.

25. The image orientation identification system in claim 21, wherein said predetermined desaturation gradient for sky comprises, from horizon to zenith, a decrease in red and green light trace components and a substantially constant blue light trace component.

26. The image orientation identification system in claim 19, wherein said color classifier:

forms a belief map of sky color pixels;

computes an adaptive threshold of sky color; and

determines if said potential sky pixels exceed said threshold.

27. The image orientation identification system in claim 26, wherein said color classifier computes said adaptive threshold by identifying a first valley on a belief histogram derived from said belief map.

28. The image orientation identification system in claim 27, wherein belief histogram and said belief map are unique to said image.

29. A method of using and a computer program running on a computer system to determine image orientation, said method comprising:

using said computer program to classify sky pixels in said image by color;

using said computer program to compute desaturation gradients of said sky pixels;

using said computer program to classify said image as one of portrait and landscape based on average absolute values of horizontal and vertical desaturation gradient of said sky pixels; and

using said computer program to determine orientation of said image based on said average horizontal and vertical desaturation gradients.

30. The method in claim 29, further comprising:

using said computer program to identify spatially contiguous regions of said sky pixels; and

using said computer program to identify actual sky regions by eliminating ones of said spatially contiguous regions that have a texture above a predetermined texture threshold.

31. The method in claim 30, further comprising using said computer program to confirm that said actual sky regions are true sky regions by comparing said desaturation gradients with a predetermined desaturation gradient for sky.

32. The method in claim 30, further comprising using said computer program to maintain only ones of said spatially contiguous regions as said actual sky regions which:

have substantially dissimilar average horizontal and vertical desaturation gradients;

have a color distribution which is consistent with a predetermined color distribution for sky; and

contact a border of said image.

33. The method in claim 29, wherein said using said computer program to classify said image as one of portrait and landscape includes:

using said computer program to classify said image as said portrait if an absolute value of said average horizontal desaturation gradient is greater than an absolute value of said average vertical desaturation gradient; and

using said computer program to classify said image as said landscape if an absolute value of said average horizontal desaturation gradient is less than an absolute value of said average vertical desaturation gradient.

34. The method in claim 29, wherein said desaturation gradients comprise desaturation gradients for red, green and blue trace components of said image.

35. The method in claim 31, wherein said predetermined desaturation gradient for sky comprises, from horizon to zenith, a decrease in red and green light trace components and a substantially constant blue light trace component.

36. The method in claim 29, wherein said using said computer program to classify sky pixels in said image by color comprises:

using said computer program to form a belief map of sky color pixels;

using said computer program to compute an adaptive threshold of sky color; and

using said computer program to determine if said sky color pixels exceed said threshold.

37. The method in claim 36, wherein said using said computer program to compute said adaptive threshold comprises using said computer program to identify a first valley in a belief histogram derived from on said belief map.

38. The method in claim 37, wherein said belief histogram and said belief map are unique to said image.