Detecting recurring themes in consumer image collections

• US 9,042,646 B2
• Filed: 01/03/2014
• Issued: 05/26/2015
• Est. Priority Date: 08/30/2011
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Das, Madirakshi, Loui, Alexander C.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Harandi, Siamak
• Application Number:
  US14/146,886
• Publication Number:
  US 20140119652A1
• Time in Patent Office: 508 Days
• Field of Search: None
• US Class Current: 382/170
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A method of identifying groups of related digital images in a digital image collection, comprising: analyzing each of the digital images to generate associated feature descriptors related to image content or image capture conditions; storing the feature descriptors associated with the digital images in a metadata database; automatically analyzing the metadata database to identify a plurality of frequent itemsets, wherein each of the frequent itemsets is a co-occurring feature descriptor group that occurs in at least a predefined fraction of the digital images; determining a probability of occurrence for each the identified frequent itemsets; determining a quality score for each of the identified frequent itemsets responsive to the determined probability of occurrence; ranking the frequent itemsets based at least on the determined quality scores; and identifying one or more groups of related digital images corresponding to one or more of the top ranked frequent itemsets.

Claims

1. A method comprising:

storing, in a computing device, feature descriptors for each image in a first plurality of images, wherein the feature descriptors are related to image content or image capture conditions;

identifying, by the computing device, a plurality of itemsets, wherein each itemset of the plurality of itemsets comprises a set of co-occurring feature descriptors that occurs in a respective subset of images in the first plurality of images;

associating, by the computing device, each itemset with the respective subset of images in which the itemset occurs;

determining, by the computing device, a quality score for each itemset, wherein the quality score is related to a probability that the itemset occurs in a general population of images based on a probability distribution determined from an analysis of a second plurality of images;

identifying, by the computing device, a first itemset based on the determined quality score; and

storing, by the computing device, an indication of a respective subset of images associated with the identified first itemset.

2. The method of claim 1, further comprising:

performing an event clustering operation to cluster the first plurality of images into a set of events;

determining a confidence score for each identified itemset based at least upon the events associated with the subset of images corresponding to the identified itemset;

removing, from the plurality of itemsets, itemsets having an associated confidence score lower than a pre-defined threshold.

3. The method of claim 2, wherein the confidence score for an itemset is based on a comparison of the number of images from a given event that are associated with the itemset to the total number of images from the event.

4. The method of claim 1, wherein the feature descriptors are stored using a transactional database format.

5. The method of claim 1, further comprising determining the feature descriptors for the images in the first plurality of images by analyzing image content of the images.

6. The method of claim 5, wherein the feature descriptors are determined by analyzing image content of the images, and wherein the image content comprises scene class features, materials class features, low-level features, or facial features.

7. The method of claim 1, further comprising determining the feature descriptors for the images in the first plurality of images by analyzing capture metadata associated with the images.

8. The method of claim 7, wherein the feature descriptors are determined by analyzing capture metadata that indicates capture condition features.

9. The method of claim 1, wherein the feature descriptors include feature descriptors corresponding to metadata tags associated with the images.

10. The method of claim 9, further comprising determining the metadata tags by automatically analyzing image content or capture metadata associated with the first plurality of images.

11. The method of claim 1, wherein the quality score of each respective itemset is inversely related to the probability that the itemset occurs in a general population of images.

12. The method of claim 1, wherein the feature descriptors are quantized into a finite number of predefined states.

13. The method of claim 1, wherein identifying the first itemset is also based upon a time difference between an earliest capture time and a latest capture time associated with the subset of images corresponding to the first itemset.

14. The method of claim 1, wherein identifying the first itemset is also based upon the number of people in the subset of images corresponding to the first itemset.

15. The method of claim 1, further comprising causing display of the subset of images associated with the first itemset in the form of a slideshow or photo-book.

16. The method of claim 1, wherein the identified first itemset is identified based on the first itemset having a lower probability that the itemset occurs in a general population of images than other identified itemsets.

17. A computing system comprising:

a memory configured to store feature descriptors for each image in a first plurality of images, wherein the feature descriptors are related to image content or image capture conditions; and

a processing system configured to;

identify a plurality of itemsets, wherein each itemset of the plurality of itemsets comprises a set of co-occurring feature descriptors that occurs in a respective subset of images in the first plurality of images;

associate each itemset with the respective subset of images in which the itemset Occurs;

determine a quality score for each respective itemset, wherein the quality score is related to a probability that the itemset occurs in a general population of images based on a probability distribution determined from an analysis of a second plurality of images;

identify a first itemset based on the determined quality score; and

store, in the memory, an indication of a respective subset of images associated with the identified first itemset.

18. The system of claim 17, wherein the processing system is further configured to:

perform an event clustering operation to cluster the first plurality of images into a set of events;

determine a confidence score for each identified itemset based at least in part on the number of images from the same events in the digital images corresponding to the frequent itemset;

remove itemsets having an associated confidence score lower than a pre-defined threshold from the first plurality of itemsets.

19. The system of claim 17, wherein the processing system is further configured to determine the feature descriptors by analyzing image content of each of the first plurality of images, wherein the image content comprises scene class features, materials class features, low-level features, and facial features.

20. A non-transitory computer readable medium having stored thereon instructions executable by a computing device to cause the computing device to perform functions, the functions comprising:

storing feature descriptors for each image in a first plurality of images, wherein the feature descriptors are related to image content or image capture conditions;

identifying a plurality of itemsets, wherein each of the identified itemsets is a set of co-occurring feature descriptors that occurs in a respective subset of images in the first plurality of images;

associating each itemset with the respective subset of images in which the itemset occurs;

determining a quality score for each respective itemset, wherein the quality score is related to a probability that the itemset occurs in a general population of images based on a probability distribution determined from an analysis of a second plurality of images;

identifying a first itemset based on the determined quality score; and

storing an indication of a respective subset of images associated with the identified first itemset.