Photo classification using optical parameters of camera from EXIF metadata

• US 8,094,948 B2
• Filed: 04/25/2008
• Issued: 01/10/2012
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Jain, Ramesh, Sinha, Pinaki
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): SHAH, UTPAL D
• Application Number:
  US12/110,065
• Publication Number:
  US 20080292196A1
• Time in Patent Office: 1,355 Days
• Field of Search: 382/224, 382/225
• US Class Current: 382/225
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A method of classifying and organizing digital images utilizing optical metadata (captured using multiple sensors on the camera) may define semantically coherent image classes or annotations. The method defines optical parameters based on the physics of vision and operation of a camera to cluster related images for future search and retrieval. An image database constructed using photos taken by at least thirty different users over a six year period on four different continents was tested using algorithms to construct a hierarchal clustering model to cluster related images. Additionally, a survey about the most frequent image classes shot by common people forms a baseline model for automatic annotation of images for search and retrieval by query keyword.

Claims

1. A method for classifying digital images, comprising:

hierarchically clustering the digital images into a set of meaningful clusters based on at least two levels of clustering optical parameters, including;

clustering the digital images based on determined light content of the digital images and

clustering the digital images based on depth of field associated with the digital images;

associating the set of meaningful clusters to a set of associated classes used by a user; and

classifying the digital images according to the set of associated classes.

2. The method for classifying digital images of claim 1, further comprising:

analyzing the optical parameters for context information and content information.

3. The method for classifying digital images of claim 1, further comprising:

automatically annotating each digital image with a set of text for further classification.

4. The method for classifying digital images of claim 1, wherein:

the optical parameters are gathered from EXIF data attached to metadata of the digital images.

5. The method for classifying digital images of claim 1, further comprising:

determining an associated classification for each of the digital images based on the probability that a block vocabulary and a word vocabulary apply.

6. The method for classifying digital images of claim 5, further comprising:

retrieving digital images searched by a query tag based on the word vocabulary.

7. The method for classifying digital images of claim 1, further comprising:

retrieving images having a tag a probability higher than a predetermined cutoff.

8. A method for organizing digital images, comprising:

deriving optical parameters from the digital images;

determining a visual cue in the digital images based on depth of field associated with the digital images;

accessing a set of subject classes commonly used by a user and assembled into predefined parameters;

determining a semantic concept for the digital image by associating the visual cue with the set of digital image subject classes; and

organizing the digital images into classifications according to the associations determined by the visual cue.

9. The method for organizing digital images of claim 8, further comprising:

automatically annotating the digital images by associating the classifications with tags.

10. The method for organizing digital images of claim 8, further comprising:

analyzing the optical parameters for context information and content information.

11. The method for organizing digital images of claim 8, further comprising:

retrieving images having a tag corresponding to a query tag, where the tag has a probability higher than a predetermined cutoff.

12. The method for organizing digital images of claim 8, further comprising:

determining an associated classification for each of the digital images based on the probability that a block vocabulary and a word vocabulary apply.

13. The method for organizing digital images of claim 12, further comprising:

retrieving digital images searched by a query tag using the block vectors associated with the probability that the block vocabulary and the word vocabulary apply.

14. The method for organizing digital images of claim 8, further comprising:

manually assigning text tags to images according to the classifications the images are associated with.

15. A method for using optical metadata of a digital image to classify the digital image, the method comprising:

analyzing the optical metadata to cluster the digital image with other digital images having similar optical concepts based on indicators of light content and depth of field in the optical meta data of the digital image;

comparing the cluster with human induced labeled images having similar optical concepts based on indicators of light content and depth of field associated with the human induced labeled images; and

corresponding the human induced labeled images with a classification for the digital image.

16. The method of claim 15, further comprising:

automatically annotating the digital image by associating the classification with a tag.

17. The method of claim 16, wherein:

the automatic annotation includes associating the digital image by classification according to extracted optical context data from the optical metadata.

18. The method of claim 15, further comprising:

retrieving images having a tag corresponding to a query tag, where the tag has a probability higher than a predetermined cutoff.