Method for detecting subject matter regions in images

• US 7,062,085 B2
• Filed: 09/13/2001
• Issued: 06/13/2006
• Est. Priority Date: 09/13/2001
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo, Singhal, Amit
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US09/951,256
• Publication Number:
  US 20030053686A1
• Time in Patent Office: 1,734 Days
• Field of Search: 382162-167, 382224-228, 382199-203, 382168-170
• US Class Current: 382/165
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/168 involving transform domain methods
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/38 Outdoor scenes

Abstract

Disclosed is a method for detecting subject matter regions in a digital color image having pixels of (red, green, blue) values, comprising the steps of: assigning to each pixel a belief value as belonging to the subject matter region based on color and texture features; forming spatially contiguous candidate subject matter regions by thresholding the belief values; analyzing the spatially contiguous regions based on one or more unique characteristics of the subject matter to determine the probability that a region belongs to the subject matter; and generating a map of detected subject matter regions and associated probability that the regions belongs to the subject matter.

Claims

1. A method for detecting subject matter regions in a digital color image having pixels of (red, green, blue) values, comprising the steps of:

a) assigning to each pixel a belief value as belonging to the subject matter region based on color and texture features;

b) forming spatially contiguous candidate subject matter regions by thresholding the belief values;

c) analyzing the spatially contiguous regions based on one or more unique characteristics of the subject matter to determine the probability that a region belongs to the subject matter; and

d) generating a map of detected subject matter regions and associated probability that the regions belongs to the subject matter.

2. The method claimed in claim 1, wherein the subject matter is cloudy or overcast sky and step of analyzing the spatially contiguous regions includes the steps of:

c1) computing an open space map containing spatially contiguous regions of low activity;

c2) comparing the spatially contiguous regions of low activity with the spatially contiguous candidate subject matter regions;

c3) labeling those candidate regions that have significant overlap with the low activity regions as subject matter regions; and

c4) labeling those candidate regions that do not have any overlap with the low activity regions as subject matter regions.

3. The method claimed in claim 1, wherein the subject matter is cloudy sky and step of assigning to each pixel a belief value includes the steps of:

a1) calculating one luminance value and two chrominance values from the (red, green, blue) values of each pixel in the digital color image;

a2) calculating a normalized luminance feature as a ratio of the luminance value and a maximum luminance value of the entire digital color image; and

a3) calculating two chrominance features as equal to the two chrominance values.

4. The method claimed in claim 1, wherein the step of assigning to each pixel a belief value includes the steps of:

a1) calculating a luminance value from (red, green, blue) values of each pixel in the digital color image; and

a2) calculating a plurality of texture features using the luminance values.

5. The method claimed in claim 4, wherein the texture features are coefficients derived from a wavelet transform based on multiresolution analysis of the digital color image.

6. The method claimed in claim 1, wherein the step of forming spatially contiguous candidate subject matter regions by thresholding the belief values includes the steps of:

b1) producing a subject matter belief map using subject matter belief values of all the pixels in the image;

b2) computing gradient values from the subject matter belief map;

b3) selecting pixels having gradient values greater than a predetermined value;

b4) forming a histogram of subject matter belief values using only the selected pixels from the previous step;

b5) locating a valley point in the histogram; and

b6) determining the subject matter belief threshold as the subject matter belief value at the located valley point of the histogram.

7. The method claimed in claim 1, wherein the subject matter is lawn grass, snow field, sand or standing water and step of analyzing the spatially contiguous regions includes the steps of:

c1) computing an open space map containing spatially contiguous regions of low activity;

c2) comparing the spatially contiguous regions of low activity with the spatially contiguous candidate subject matter regions; and

c3) labeling those candidate regions that have significant overlap with the low activity regions as subject matter regions.

8. The method claimed in claim 2, wherein the step of assigning the pixel belief value is performed with a trained neural network.

9. The method claimed in claim 7, wherein the step of assigning the pixel belief value is performed with a trained neural network.

10. The method claimed in claim 1, further comprising the step of:

e) performing image classification using the map of detected subject matter regions.

11. The method claimed in claim 1, further comprising the step of:

e) determining image orientation using the map of detected subject matter regions.

12. The method claimed in claim 1, further comprising the step of:

e) automatically generating album pages having similar subject matter using the map of detected subject matter regions.

13. The method claimed in claim 1, further comprising the step of:

e) determining a main subject of the image using the map of detected subject matter regions.

14. The method claimed in claim 1, further comprising the step of:

e) determining background of the image using the map of detected subject matter regions.

15. The method claimed in claim 1, further comprising the step of:

e) retrieving images having similar subject matter from an image data base using the map of detected subject matter regions.

16. The method claimed in claim 1, further comprising the step of:

e) selectively compressing the image using the map of detected subject matter regions.

17. A system for detecting subject matter regions in a digital color image having pixels of (red, green, blue) values, comprising:

a) means for assigning to each pixel a belief value as belonging to the subject matter region based on color and texture features;

b) means for forming spatially contiguous candidate subject matter regions by thresholding the belief values;

c) means for analyzing the spatially contiguous regions based on one or more unique characteristics of the subject matter to determine the probability that a region belongs to the subject matter; and

d) means for generating a map of detected subject matter regions and associated probability that the regions belongs to the subject matter.

18. The system claimed in claim 17, wherein the subject matter is cloudy or overcast sky and the means for analyzing the spatially contiguous regions includes:

c1) means for computing an open space map containing spatially contiguous regions of low activity;

c2) means for comparing the spatially contiguous regions of low activity with the spatially contiguous candidate subject matter regions;

c3) means for labeling those candidate regions that have significant overlap with the low activity regions as subject matter regions; and

c4) means for labeling those candidate regions candidate regions that do not have any overlap with the low activity regions as subject matter regions.

19. The system claimed in claim 17, wherein the subject matter is cloudy sky and the means for assigning to each pixel a belief value includes:

a1) means for calculating one luminance value and two chrominance values from the (red, green, blue) values of each pixel in the digital color image;

a2) means for calculating a normalized luminance feature as a ratio of the luminance value and a maximum luminance value of the entire digital color image; and

a3) means for calculating two chrominance features as equal to the two chrominance values.

20. The system claimed in claim 17, wherein the means for assigning to each pixel a belief value includes:

a1) means for calculating a luminance value from (red, green, blue) values of each pixel in the digital color image; and

a2) means for calculating a plurality of texture features using the luminance values.

21. The system claimed in claim 20, wherein the texture features are coefficients derived from a wavelet transform based on multiresolution analysis of the digital color image.

22. The system claimed in claim 17, wherein the means for forming spatially contiguous candidate subject matter regions by thresholding the belief values includes:

b1) means for producing a subject matter belief map using subject matter belief values of all the pixels in the image;

b2) means for computing gradient values from the subject matter belief map;

b3) means for selecting pixels having gradient values greater than a predetermined value;

b4) means for forming a histogram of subject matter belief values using only the selected pixels from the previous step;

b5) means for locating a valley point in the histogram; and

b6) means for determining the subject matter belief threshold as the subject matter belief value at the located valley point of the histogram.

23. The system claimed in claim 17, wherein the subject matter is lawn grass, snow field, sand, or standing water and the means for analyzing the spatially contiguous regions includes:

c1) means for computing an open space map containing spatially contiguous regions of low activity;

c2) means for comparing the spatially contiguous regions of low activity with the spatially contiguous candidate subject matter regions; and

c3) means for labeling those candidate regions that have significant overlap with the low activity regions as subject matter regions.

24. The system claimed in claim 17, wherein the means for assigning the pixel belief value is a trained neural network.

25. The system claimed in claim 23, wherein the means for assigning the pixel belief value is a trained neural network.

26. The system claimed in claim 17, further comprising:

e) means for performing image classification using the map of detected subject matter regions.

27. The system claimed in claim 17, further comprising:

e) means for determining image orientation using the map of detected subject matter regions.

28. The system claimed in claim 17, further comprising:

e) means for automatically generating album pages having similar subject matter using the map of detected subject matter regions.

29. The system claimed in claim 17, further comprising:

e) means for determining a main subject of the image using the map of detected subject matter regions.

30. The system claimed in claim 17, further comprising:

e) means for determining background of the image using the map of detected subject matter regions.

31. The system claimed in claim 17, further comprising:

e) means for retrieving images having similar subject matter from an image data base using the map of detected subject matter regions.

32. The system claimed in claim 17, further comprising:

e) means for selectively compressing the image using the map of detected subject matter regions.