Using camera metadata to classify images into scene type classes

• US 7,668,369 B2
• Filed: 04/26/2006
• Issued: 02/23/2010
• Est. Priority Date: 04/26/2006
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Tretter, Daniel, Yen, Jonathan, Wu, Peng
• Primary Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US11/411,481
• Publication Number:
  US 20070253699A1
• Time in Patent Office: 1,399 Days
• Field of Search: 382168-170, 382224-228, 382/162, 382/165, 3482311-23199, 345/589
• US Class Current: 382/170
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

In one aspect, a metadata-based classification result is obtained based on metadata associated with an image. A histogram-based classification result is determined based on a histogram of intensity values derived from the image. The image is classified into a scene type class based on the metadata-based classification result and the histogram-based classification result. In another aspect, a first condition is applied on a first one of metadata associated with an image corresponding to a measure of amount of light received in capturing the image to obtain a first metadata-based classification result. A second condition is applied on a second one of the metadata corresponding to a measure of brightness of the image to obtain a second metadata-based classification result. The image is classified into a scene type class based on the first and second metadata-based classification results.

Claims

1. An image processing method, comprising:

obtaining a metadata-based classification result based on metadata associated with an image, wherein the obtaining comprises initially classifying the image into the scene type class based on a first condition on a first one of the metadata corresponding to a measure of amount of light received in capturing the image and a second condition on a second one of the metadata corresponding to a measure of brightness of the image;

determining a histogram-based classification result based on a histogram of intensity values derived from the image wherein the determining comprises applying to the histogram a histogram predicate defining the scene type class in terms of multiple conditions on the histogram; and

classifying the image into a scene type class based on the metadata-based classification result and the histogram-based classification result;

wherein the obtaining, the determining, and the classifying are performed by computing hardware.

2. The method of claim 1, wherein the obtaining comprises applying to the metadata a metadata predicate defining the scene type class in terms of at least one condition on the metadata.

3. The method of claim 1, wherein the obtaining comprises performing an initial classification of the image into the scene type class based on at least one condition on the metadata and the determining comprises verifying the initial classification of the image into the scene type class based on at least one condition on the histogram.

4. An image processing method, comprising:

obtaining a metadata-based classification result based on metadata associated with an image, wherein the obtaining comprises performing an initial classification of the image into a scene type class based on at least one condition on the metadata, the obtaining comprises initially classifying the image into the scene type class based on a first condition on a first one of the metadata corresponding to a measure of amount of light received in capturing the image and a second condition on a second one of the metadata corresponding to a measure of brightness of the image, and the obtaining comprises initially classifying the image into a night scene class in response to a determination that the first metadata exceeds a first threshold value and the second metadata is below a second threshold value;

determining a histogram-based classification result based on a histogram of intensity values derived from the image, wherein the determining comprises verifying the initial classification of the image into the scene type class based on at least one condition on the histogram; and

classifying the image into the scene type class based on the metadata-based classification result and the histogram-based classification result;

wherein the obtaining, the determining, and the classifying are performed by computing hardware.

5. The method of claim 4, wherein the verifying comprises confirming the initial classification of the image into the night scene class based at least in pan on a determination that at least one portion of the histogram has a respective property in relation to other portions of the histogram.

6. The method of claim 5, wherein the confirming comprises dividing at least a section of the histogram into different intensity portions ranging from a highest intensity portion to a lowest intensity portion, and determining whether the highest intensity portion of the histogram has an average count that is lower than average frequencies of at least most of the other intensity portions of the histogram.

7. The method of claim 6, wherein the confirming additionally comprises determining whether the average count of the highest intensity portion is below a threshold count value.

8. The method of claim 7, wherein the confirming additionally comprises determining whether a measure of average intensity of the image is below an average intensity threshold.

9. An image processing system, comprising:

memory storing computer process instructions; and

computing hardware coupled to the memory, operable to execute the instructions, and based on the execution of the instructions operable to perform operations comprising

implementing a metadata-based scene type classifier that performs an initial classification of an image into a scene type class based on metadata associated with the image, wherein the metadata-based scene type classifier initially classifies the image into the scene type class based on a first condition in a first one of the metadata corresponding to a measure of amount of light received in capturing the image and a second condition on a second one of the metadata corresponding to a measure of brightness of the image, and

implementing a verification classifier that verifies the initial classification of the image into the scene type class based on application of a histogram predicate to a histogram of intensity values derived from the image, wherein the histogram predicate defines the scene type class in terms of multiple conditions on the histogram.

10. The system of claim 9, wherein the metadata-based scene type classifier initially classifies the image into a night scene class in response to a determination that the first metadata exceeds a first threshold value and the second metadata is below a second threshold value.

11. The system of claim 10, wherein the verification classifier confirms the initial classification of the image into the night scene class based at least in pan on a determination that at least one portion of the histogram has a respective property in relation to other portions of the histogram.

12. The system of claim 11, wherein the verification classifier divides at least a section of the histogram into different intensity portions ranging from a highest intensity portion to a lowest intensity portion, and determines whether the highest intensity portion of the histogram has an average count that is lower than average frequencies of at least most of the other intensity portions of the histogram.

13. The system of claim 12, wherein the verification classifier additionally determines whether the average count of the highest intensity portion is below a threshold count value.

14. The system of claim 13, wherein the verification classifier additionally determines whether a measure of average intensity of the image is below an average intensity threshold.

15. An image processing method, comprising:

applying a first condition on a first one of metadata associated with an image to obtain a first metadata-based classification result, wherein the first metadata corresponds to a measure of amount of light received in capturing the image;

applying a second condition on a second one of the metadata corresponding to a measure of brightness of the image to obtain a second metadata-based classification result; and

classifying the image into a scene type class based on the first and second metadata-based classification results;

wherein the applying of the first condition, the applying of the second condition, and the classifying are performed by computing hardware.

16. The method of claim 15, wherein applying the first condition comprises determining whether the first metadata exceeds a first threshold value, applying the second condition comprises determining whether the second metadata is below a second threshold value, and the classifying comprises classifying the image into a night scene class in response to determinations that the first metadata exceeds the first threshold value and the second metadata is below the second threshold value.

17. The method of claim 15, wherein applying the first condition comprises determining whether the first metadata is below a first threshold value, applying the second condition comprises determining whether the second metadata exceeds a second threshold value, and the classifying comprises classifying the image into a snow scene class in response to determinations that the first metadata is below the first threshold value and the second metadata exceeds the second threshold value.