Detecting Recurring Events in Consumer Image Collections

• US 20140105492A1
• Filed: 12/18/2013
• Published: 04/17/2014
• Est. Priority Date: 08/25/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Das, Madirakshi, Loui, Alexander C.
• Granted Patent: US 8,811,755 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene

Abstract

A method of detecting recurring events in a digital image collection taken over a pre-determined period of time is disclosed. The method uses a processor for analyzing the digital image collection to produce a two-dimensional representation of the distribution of image capture activity over time and detecting recurring events by identifying spatial clusters in the two-dimensional representation.

Claims

1. A method to detect recurring events in a digital image collection, comprising:

identifying events in a collection of digital images, each event comprising at least one digital image, each digital image having capture time metadata;

characterizing the identified events in a multi-dimensional representation including a representation having a first time axis and a second time axis that is different from the first time axis based, at least in part, on a distribution of the identified events'"'"' digital images'"'"' capture times; and

identifying one or more recurring events from the identified events based, at least in part, on a spatial clustering analysis of the multi-dimensional representation of the events.

4. The method of claim 1, wherein the act of identifying events comprises identifying events based, at least in part, on foreground and background segmentation of digital images in the digital image collection.

5. The method of claim 1, wherein the act of identifying one or more recurring events comprises, at least in part, applying an event signature-based filter to the identified events.

6. The method of claim 5, wherein the act of identifying one or more recurring events comprises applying a spatial clustering analysis to only those identified events that pass the event signature-based filter.

2. The method of claim 2, wherein the act of identifying events comprises identifying events based, at least in part, on color content of the digital images in the digital image collection.

3. The method of claim 2, wherein the act of identifying events based, at least in part, on content of the digital images comprises:

partitioning a first group of the digital images into a first plurality of blocks;

determining a color histogram for at least some of the first group'"'"'s digital images;

partitioning a second group of the digital images into a second plurality of blocks, wherein the capture time of each digital image of the second group is later than the capture time of any of the digital images of the first group;

determining a color histogram for each digital image of the second group that corresponds to a block of the first digital image; and

comparing the color histograms of the first digital image'"'"'s blocks with the corresponding color histograms of the second digital image'"'"'s blocks.

7. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

identify events in a collection of digital images, each event comprising at least one digital image, each digital image having capture time metadata;

characterize the identified events in a multi-dimensional representation including a representation having a first time axis and a second time axis that is different from the first time axis based, at least in part, on a distribution of the identified events'"'"' digital images'"'"' capture times; and

identify one or more recurring events from the identified events based, at least in part, on a spatial clustering analysis of the multi-dimensional representation of the events.

8. The non-transitory program storage device of claim 7, wherein the instructions to cause one or more processors to identify events comprise instructions to cause one or more processors to identify events based, at least in part, on color content of the digital images in the digital image collection.

9. The non-transitory program storage device of claim 8, wherein the instructions to cause one or more processors to identify events based, at least in part, on content of the digital images comprise instructions to cause one or more processors to:

partition a first group of the digital images into a first plurality of blocks;

determine a color histogram for at least some of the first group'"'"'s digital images;

partition a second group of the digital images into a second plurality of blocks, wherein the capture time of each digital image of the second group is later than the capture time of any of the digital images of the first group;

determine a color histogram for each digital image of the second group that corresponds to a block of the first digital image; and

compare the color histograms of the first digital image'"'"'s blocks with the corresponding color histograms of the second digital image'"'"'s blocks.

10. The non-transitory program storage device of claim 7, wherein the instructions to cause one or more processors to identify events further comprise instructions to cause one or more processors to identify sub-events among the digital images having identified events.

11. The non-transitory program storage device of claim 7, wherein the instructions to cause one or more processors to identify events comprise instructions to cause one or more processors to identify events based, at least in part, on foreground and background segmentation of the digital images in the digital image collection.

12. The non-transitory program storage device of claim 7, wherein the instructions to cause one or more processors to characterize the identified events comprise instructions to cause one or more processors to characterize the identified events based, at least in part, on a two dimensional distribution of the identified events'"'"' digital images'"'"' capture times.

13. The non-transitory program storage device of claim 7, wherein the instructions to cause one or more processors to identify one or more recurring events comprise instructions to cause one or more processors to apply an event signature-based filter to the identified events.

14. The non-transitory program storage device of claim 13, wherein the instructions to cause one or more processors to identify one or more recurring events comprise instructions to cause one or more processors to apply a spatial clustering analysis to only those identified events that pass the event signature-based filter.

15. The non-transitory program storage device of claim 13, wherein the event signature-based filter is based, at least in part, on a Bayesian belief network-based event classification.

16. A computer system, comprising:

memory;

a display communicatively coupled to the memory; and

one or more processors operatively coupled to the memory and display and configured to execute instructions stored in the memory to

identify events in a collection of digital images stored in the memory, each event comprising at least one digital image, each digital image having capture time metadata,

characterize the identified events in a multi-dimensional representation including a representation having a first time axis and a second time axis that is different from the first time axis based, at least in part, on a distribution of the identified events'"'"' digital images'"'"' capture times, and

identify one or more recurring events from the identified events based, at least in part, on a spatial clustering analysis of the multi-dimensional representation of the events.

17. The computer system of claim 16, further comprising instructions stored in the memory to cause the one or more processors to identify at least one of the identified events and the event'"'"'s corresponding digital images on the display.