PHOTO CLASSIFICATION USING OPTICAL PARAMETERS OF CAMERA FROM EXIF METADATA

• US 20080292196A1
• Filed: 04/25/2008
• Published: 11/27/2008
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: Sinha, Pinaki, Jain, Ramesh
• Granted Patent: US 8,094,948 B2
• Time in Patent Office: Days
• US Class Current: 382/225
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A method of classifying and organizing digital images utilizing optical metadata (captured using multiple sensors on the camera) may define semantically coherent image classes or annotations. The method defines optical parameters based on the physics of vision and operation of a camera to cluster related images for future search and retrieval. An image database constructed using photos taken by at least thirty different users over a six year period on four different continents was tested using algorithms to construct a hierarchal clustering model to cluster related images. Additionally, a survey about the most frequent image classes shot by common people forms a baseline model for automatic annotation of images for search and retrieval by query keyword.

Claims

1. A method for classifying digital images, the method comprising:

clustering optical parameters of the digital images into a set of meaningful clusters;

associating the set of meaningful clusters to a set of associated classes used by a user; and

classifying the digital images according to the set of associated classes.

2. The method for classifying digital images of claim 1, further comprising:

analyzing the optical parameters for context information and content information.

3. The method for classifying digital images of claim 1, further comprising:

automatically annotating each digital image with a set of text for further classification.

4. The method for classifying digital images of claim 1, wherein:

the optical parameters are gathered from EXIF data attached to metadata of the digital images.

5. The method for classifying digital images of claim 1, further comprising:

assembling together a set of block vocabulary;

assembling together a set of word vocabulary;

applying an algorithm to the digital images to predict probabilities where members of the block and word vocabulary are associated to the digital images; and

associating the probabilities into block vectors for determining classification of the images into the associated classes.

6. The method for classifying digital images of claim 5, further comprising:

providing a searchable environment using query tags for the digital images; and

retrieving digital images searched by query tag using the block vectors.

7. The method for classifying digital images of claim 1, further comprising:

providing a searchable environment using query tags for the digital images; and

retrieving images having the query tags with a probability higher than a predetermined cutoff.

8. A method for organizing digital images, the method comprising:

deriving optical parameters from the digital images;

accessing a set of subject classes commonly used by a user and assembled into predefined parameters;

determining what the user was trying to capture in the digital image by associating the derived optical parameters with the set of digital image subject classes; and

organizing the digital images into classifications according to the associations determined by the derived optical parameters.

9. The method for organizing digital images of claim 8, further comprising:

accessing a file of constructed tags; and

automatically annotating the digital images by associating the classifications with the tags.

10. The method for organizing digital images of claim 8, further comprising:

analyzing the optical parameters for context information and content information.

11. The method for organizing digital images of claim 8, further comprising:

providing a searchable environment using query tags for the digital images; and

retrieving images having the query tags with a probability higher than a predetermined cutoff.

12. The method for organizing digital images of claim 8, further comprising:

assembling together a set of block vocabulary;

assembling together a set of word vocabulary;

applying an algorithm to the digital images to predict a probability where members of the block and word vocabulary are associated to the digital images; and

associating the probabilities into block vectors for determining classification of the images into the associated classes.

13. The method for organizing digital images of claim 12, further comprising:

providing a searchable environment using query tags for the digital images; and

retrieving digital images searched by query tag using the block vectors.

14. The method for organizing digital images of claim 8, further comprising:

manually assigning text tags to images according to the classifications the images are associated with.

15. A method for using optical metadata of a digital image to classify the digital image, the method comprising:

analyzing the optical metadata to find clusters of digital images having similar optical concepts;

comparing the clusters with human induced classes; and

corresponding the human induced classes with a classification for the digital image.

16. The method of claim 15, wherein:

in the step of analyzing, the clusters are derived from measurements including ambient light level data.

17. The method of claim 15, wherein:

in the step of analyzing, the clusters are derived from measurements including depth of focus data.

18. The method of claim 15, further comprising:

accessing a file of constructed tags; and

automatically annotating the digital images by associating the classifications with the tags.

19. The method of claim 18, wherein:

the step of automatic annotation includes associating the digital images by classifications according to extracted optical context data from the optical metadata.

20. The method of claim 15, further comprising:

providing a searchable environment using query tags for the digital images; and

retrieving images having the query tags with a probability higher than a predetermined cutoff.