DETECTING RECURRING THEMES IN CONSUMER IMAGE COLLECTIONS

• US 20130051670A1
• Filed: 08/30/2011
• Published: 02/28/2013
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Das, Madirakshi, Loui, Alexander C.
• Granted Patent: US 8,625,904 B2
• Time in Patent Office: Days
• US Class Current: 382/170
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A method of identifying groups of related digital images in a digital image collection, comprising: analyzing each of the digital images to generate associated feature descriptors related to image content or image capture conditions; storing the feature descriptors associated with the digital images in a metadata database; automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images; determining a probability of occurrence for each the identified frequent itemsets; determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence; ranking the frequent itemsets based at least on the determined quality scores; and identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets.

Claims

1. A method of identifying groups of related digital images in a digital image collection including a plurality of digital images, comprising:

analyzing each of the digital images in the digital image collection to generate associated feature descriptors related to image content or image capture conditions;

storing the feature descriptors associated with the digital images in a metadata database;

using a data processor to automatically analyze the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images, each frequent itemset being associated with a subset of the digital images;

determining a probability of occurrence for each the identified frequent itemsets based on one or more probability distributions determined from an analysis of a large number of image collections;

determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence;

ranking the frequent itemsets based at least on the determined quality scores;

identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets; and

storing an indication of the identified groups of related digital images in a processor-accessible memory.

2. The method of claim 1 further including removing a subset of the frequent itemsets from the plurality of frequent itemsets by:

automatically analyzing the digital images using an event clustering operation to cluster the digital images into a set of events;

determining a confidence score for each of the frequent itemsets based at least upon the events associated with the digital images corresponding to the frequent itemset;

identifying any frequent itemsets having an associated confidence score lower than a pre-defined threshold; and

removing the identified frequent itemsets from the plurality of frequent itemsets.

3. The method of claim 2 wherein the confidence score for an itemset is determined responsive to the number of images from the same events in the digital images corresponding to the frequent itemset.

4. The method of claim 1 wherein the metadata database is stored using a transactional database format.

5. The method of claim 1 wherein the feature descriptors generated for the digital images include feature descriptors determined by analyzing image content of the digital images.

6. The method of claim 5 wherein the feature descriptors determined by analyzing image content of the digital images include scene class features, materials class features, low-level features or facial features.

7. The method of claim 1 wherein the feature descriptors generated for the digital images include feature descriptors determined by analyzing capture metadata associated with the digital images.

8. The method of claim 7 wherein the feature descriptors determined by analyzing image content of the digital images include time-based features or capture condition features.

9. The method of claim 1 wherein the feature descriptors generated for the digital images include feature descriptors corresponding to metadata tags associated with the digital images.

10. The method of claim 9 wherein the metadata tags were manually supplied by a user.

11. The method of claim 9 wherein the metadata tags were determined by automatically analyzing image content or capture metadata associated with the digital images.

12. The method of claim 1 wherein the feature descriptors are quantized into a finite number of predefined states.

13. The method of claim 1 wherein the ranking of the frequent itemsets is also based upon a time difference between an earliest capture time and a latest capture time associated with the digital images corresponding to the frequent itemset.

14. The method of claim 1 wherein the ranking of the frequent itemsets is also based upon the number of people in the digital images corresponding to the frequent itemset.

15. The method of claim 1 wherein the detected related images are displayed to a user in the form of a slideshow or photo-book.

16. A system comprising:

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for identifying groups of related digital images in a digital image collection including a plurality of digital images, wherein the method comprises;

analyzing each of the digital images in the digital image collection to generate associated feature descriptors related to image content or image capture conditions;

storing the feature descriptors associated with the digital images in a metadata database;

automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images, each frequent itemset being associated with a subset of the digital images;

determining a probability of occurrence for each the identified frequent itemsets based on one or more probability distributions determined from an analysis of a large number of image collections;

determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence;

ranking the frequent itemsets based at least on the determined quality scores;

identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets; and

storing an indication of the identified groups of related digital images in a processor-accessible memory.