METHOD FOR EVENT-BASED SEMANTIC CLASSIFICATION

• US 20100124378A1
• Filed: 11/19/2008
• Published: 05/20/2010
• Est. Priority Date: 11/19/2008
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Loui, Alexander C., Wood, Mark D., Das, Madirakshi
• Granted Patent: US 8,611,677 B2
• Time in Patent Office: Days
• US Class Current: 382/225
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method of automatically classifying images in a consumer digital image collection, includes generating an event representation of the image collection; computing global time-based features for each event within the hierarchical event representation; computing content-based features for each image in an event within the hierarchical event representation; combining content-based features for each image in an event to generate event-level content-based features; and using time-based features and content-based features for each event to classify an event into one of a pre-determined set of semantic categories.

Claims

1. A method of automatically classifying images in a consumer digital image collection, comprising:

(a) generating an event representation of the image collection;

(b) computing global time-based features for each event within the hierarchical event representation;

(c) computing content-based features for each image in an event within the hierarchical event representation;

(d) combining content-based features for each image in an event to generate event-level content-based features; and

(e) using time-based features and content-based features for each event to classify an event into one of a pre-determined set of semantic categories.

2. The method of claim 1, wherein the pre-determined semantic categories include vacation, sports, family moments or social gathering.

3. The method of claim 1, wherein the global time-based features include duration of event, inter-event duration, time of day, time of year or number of images in the event.

4. The method of claim 1, wherein the content-based image-level features include people present, ages of people present, indoor or outdoor, scene type, or materials present.

5. The method of claim 1, wherein the event representation is hierarchical and includes at least two levels.

6. The method of claim 1, wherein the classification in step (e) is achieved using a probabilistic classifier, a rule-based classifier or a combination thereof.

7. The method of claim 1, wherein the method of classification in step (e) includes referencing a database of auxiliary factual information and a set of validation rules.

8. The method of claim 1, wherein the method of classification in step (e) includes referencing a database of auxiliary factual information and a set of refinement rules.

9. The method of claim 1, wherein the method of classification in step (e) includes using a rule-based system for validating and refining the output of the classifier.

10. A method of grouping time-based events detected in a collection of consumer digital images into super-events, comprising:

(a) generating time-based event boundaries;

(b) computing inter-event durations; and

(c) grouping events into clusters that are termed super-events based on the inter-event durations.

11. The method of claim 10, wherein the grouping in step c) is achieved using density-based clustering.