Detection of sky in digital color images

• US 20050147298A1
• Filed: 12/29/2003
• Published: 07/07/2005
• Est. Priority Date: 12/29/2003
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo, Gallagher, Andrew C., Hao, Wei
• Granted Patent: US 7,336,819 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method of detecting sky in a digital color image having pixels is disclosed. The method includes identifying pixels from the digital color image representing an initial sky region; developing a model based on the identified sky pixels, wherein such model is a mathematical function that has inputs of pixel position and outputs of color; and using the model to operate on the digital color image to classify additional pixels not included in the initial sky region as sky.

Claims

1. A method of detecting sky in a digital color image having pixels, the method comprising:

a) identifying pixels from the digital color image representing an initial sky region;

b) developing a model based on the identified sky pixels, wherein such model is a mathematical function that has inputs of pixel position and outputs of color; and

c) using the model to operate on the digital color image to classify additional pixels not included in the initial sky region as sky.

2. The method of claim 1 wherein the model is a two-dimensional polynomial of the pixel position in the digital color image.

3. The method of claim 2 wherein the polynomial is a second-order polynomial.

4. The method of claim 1 wherein the step of identifying an initial sky region further comprises:

i) extracting connected components of said potential sky pixels;

ii) eliminating ones of said connected components that have a texture above a predetermined texture threshold; and

iii) computing desaturation gradients of said connected components; and

comparing said desaturation gradients of said connected components with a predetermined desaturation gradient for sky to identify true sky regions in said digital color image.

5. The method of claim 1 wherein using the model to classify additional pixels as sky further comprises:

(i) determining a difference between the color of a pixel and the model estimated color for the pixel'"'"'s position; and

(ii) using the difference to determine if the pixel is in a sky region.

6. The method of claim 1 wherein using the model to classify additional pixels as sky further comprises:

(i) determining a difference between the color of a pixel and the model estimated color for the pixel'"'"'s position; and

(ii) using the difference to assign a probability that the pixel represents sky; and

(iii) using the assigned probability to determine if the pixel is in a sky region.

7. The method of claim 5 wherein fitting a model further comprises:

iv) calculating a residual error; and

v) using the difference to assign a probability that the pixel represents sky further by computing a ratio of the difference with the residual error and using the ratio to assign the probability.

8. The method of claim 5 wherein in computing the probability of the pixels as sky further comprises:

computing a color feature, related to the “

blueness”

of the pixels in the region; and

using the color feature to determine if the pixels are not sky.

9. A method of detecting sky in a digital color image having pixels, the method comprising:

a) identifying pixels from the digital color image representing an initial sky region;

b) developing a model based on the identified sky pixels, wherein such model is a mathematical function that has inputs of pixel position and outputs of color; and

c) using the model to operate on the digital color image to classify a candidate sky region not included in the initial sky region as sky.

10. The method of claim 9 wherein the model is a two-dimensional polynomial of the pixel position in the digital color image.

11. The method of claim 10 wherein the polynomial is a second-order polynomial.

12. The method of claim 9, further including calculating a model error relating to the goodness of fit;

and wherein element c) includes using the model and the model error to operate on the digital color image to classify candidate sky regions not included in the initial sky region as sky.

13. The method of claim 9, wherein element c) further comprises:

determining the size of the candidate sky region; and

using the determined size and the model to operate on the digital color image to classify the candidate sky region.