METHOD FOR EVENT-BASED SEMANTIC CLASSIFICATION

• US 20140046914A1
• Filed: 10/16/2013
• Published: 02/13/2014
• Est. Priority Date: 11/19/2008
• Status: Abandoned Application

• Current Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Das, Madirakshi, Loui, Alexander C., Wood, Mark D.
• Application Number:
  US14/054,954
• Publication Number:
  US 20140046914A1
• Time in Patent Office: Days
• US Class Current: 707/694
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method of automatically classifying images in a consumer digital image collection, includes generating an event representation of the image collection; computing global time-based features for each event within the hierarchical event representation; computing content-based features for each image in an event within the hierarchical event representation; combining content-based features for each image in an event to generate event-level content-based features; and using time-based features and content-based features for each event to classify an event into one of a pre-determined set of semantic categories.

Claims

1. A method comprising:

generating, using a processor, time-based event boundaries detected in a plurality of images;

computing inter-event durations;

grouping events into clusters based on the inter-event durations; and

validating, using a rule-based system, that each event belongs to an associated cluster based on event level content based features.

2. The method of claim 1, wherein grouping events into clusters includes using density-based clustering.

3. The method of claim 1, wherein the inter-event durations span multiple days.

4. The method of claim 1, wherein the inter-event durations span small duration gaps.

5. The method of claim 4, wherein the small duration gaps are less than 18 hours.

6. The method of claim 1, wherein validating, using a rule-based system, comprises referencing a database of auxiliary factual information associated with subjects identified in the plurality of images.

7. The method of claim 1, further comprising determining a location for each image in an event and grouping events into clusters based upon the locations.

8. The method of claim 1, further comprising determining a subject distance for each image in an event and grouping events into clusters based upon the determined locations.

9. A system comprising:

one or more processors configured to;

generate time-based event boundaries detected in a plurality of images;

compute inter-event durations;

group events into clusters based on the inter-event durations; and

validate, using a rule-based system, that each event belongs to an associated cluster based on event level content based features.

10. The system of claim 9, wherein events are grouped into clusters using density-based clustering.

11. The system of claim 9, wherein the inter-event durations span multiple days.

12. The system of claim 9, wherein the inter-event durations span less than 18 hours.

13. The system of claim 9, wherein the validation includes referencing a database of auxiliary factual information associated with subjects identified in the plurality of images.

14. The system of claim 9, wherein the one or more processors are further configured to determine a location for each image in an event and group events into clusters based upon the locations.

15. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising:

instructions to generate time-based event boundaries detected in a plurality of images;

instructions to compute inter-event durations;

instructions to group events into clusters based on the inter-event durations; and

instructions to validate, using a rule-based system, that each event belongs to an associated super-event cluster based on event level content based features.

16. The non-transitory computer-readable medium of claim 15, wherein events are grouped into clusters using density-based clustering.

17. The non-transitory computer-readable medium of claim 15, wherein the inter-event durations span multiple days.

18. The non-transitory computer-readable medium of claim 15, wherein the inter-event durations span less than 18 hours.

19. The non-transitory computer-readable medium of claim 15, wherein the instructions to validate includes instructions to reference a database of auxiliary factual information associated with subjects identified in the plurality of images.

20. The non-transitory computer-readable medium of claim 15, further comprising instructions to determine a location for each image in an event and group events into clusters based upon the locations.