Method and system for detecting semantic events

• US 20020099518A1
• Filed: 01/23/2001
• Published: 07/25/2002
• Est. Priority Date: 01/23/2001
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Qian, Richard J., Tovinkere, Vasanth R., Epshteyn, Eugene
• Granted Patent: US 6,678,635 B2
• Time in Patent Office: Days
• US Class Current: 702/179
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A method and system is provided for detecting occurrences of semantic temporal events based on observations extracted from input data and event models. The input data is fed into the system from some data source. Based on specified event to be detected, multiple-layer models corresponding to the event are retrieved. The models are used to determine the types of temporal observations to be extracted from the input data. The extracted temporal observations are then used, in combination with the multiple-layer models of the event, to detect the occurrences of the event.

Claims

1. A method for detecting a semantic temporal event, said method comprising:

retrieving multiple-layer models corresponding to said semantic temporal event;

receiving temporal observations that are extracted, from at least one data source, according to said multiple-layer models for the semantic temporal event; and

detecting one or more occurrences of the semantic temporal event based on said temporal observations and said multiple-layer models.

2. The method according to claim 1, wherein said semantic temporal event includes a sports event.

3. The method according to claim 2, wherein said sports includes a soccer game.

4. The method according to claim 1, wherein said multiple-layer models include a high level domain-specific knowledge model and a dynamic hierarchical event model.

5. The method according to claim 4, wherein said high level domain-specific knowledge includes rules of a sports game.

6. The method according to claim 4, wherein said dynamic hierarchical event model includes a hierarchical decision tree.

7. The method according to claim 4, wherein said dynamic hierarchical event model includes an entity-relationship-diagram.

8. The method according to claim 1, wherein said data source includes data acquisition devices such as a camera, a microwave sensor, and a sound recorder, and input data stream such as video, audio, text, and temporal features.

9. The method according to claim 8, wherein said temporal features include tracking position data.

10. The method according to claim 1, wherein said detecting comprises:

performing semantic temporal event detection using at least one detection method, each of said at least one detection method producing a detection result; and

combining, if more than one of said at least one detection method is used in said performing, said detection result from said at least one detection method to produce a final detection.

11. The method according to claim 10, wherein said at least one method includes dynamic Bayesian networks, rule based expert system, decision trees, Hidden Markov Models, neural networks, or fuzzy logic.

12. The method according to claim 1, further comprising:

characterizing said occurrences of the semantic temporal event, detected by said detecting, to produce a characterization; and

storing said characterization.

13. The method according to claim 12, wherein said characterization includes:

a set of statistics computed from said occurrences;

or

a set of descriptions, each of which describes an action happening in an occurrence of the semantic temporal event.

14. The method according to claim 12, further comprising:

performing temporal event prediction based on said characterization;

revising said multiple-layer models for said semantic temporal event based on said characterization; and

simulating parts of said semantic temporal event according to said characterization.

16. The system according to claim 15, further comprising:

an event characterization unit for characterizing said occurrences of the semantic temporal event, detected by said temporal event detector, to produce a characterization for the occurrences of the semantic temporal event.

17. The system according to claim 16, further comprising:

a storage mechanism for storing the characterization produced by said event characterization unit;

an event prediction unit for performing temporal event prediction based on said characterization;

an event model updating unit for modifying said multiple-layer models based on said characterization; and

an event simulation unit for simulating parts of said semantic temporal event according to said characterization.

19. The medium according to claim 18, wherein said semantic temporal event includes a sports temporal event.

20. The medium according to claim 18, wherein said sports includes a soccer game.

21. The medium according to claim 18, wherein said multiple-layer models include a high level domain-specific knowledge model and a dynamic hierarchical event model.

22. The medium according to claim 21, wherein said high level domain-specific knowledge includes rules of a sports game.

23. The medium according to claim 1, wherein said detecting comprises:

performing semantic temporal event detection using at least one detection method, each of said at least one detection method producing a detection result; and

combining, if more than one of said at least one detection method is used in said performing, said detection result from said at least one method to produce a detection.

24. The medium according to claim 23, wherein said at least one method includes dynamic Bayesian networks, rule based expert system, decision trees, Hidden Markov Models, neural networks, or fuzzy logic.

25. The medium according to claim 18, said program further comprising:

characterizing said occurrences of the semantic temporal event, detected by said detecting, to produce a characterization; and

storing said characterization.

26. The medium according to claim 25, wherein said characterization includes:

a set of statistics computed from said occurrences;

or

a set of descriptions, each of which describes an action happening in an occurrence of the semantic temporal event.

27. The medium according to claim 25, said program further comprising:

performing temporal event prediction based on said characterization;

revising said hierarchical event model based on said characterization; and

simulating parts of said semantic event according to said characterization.

29. The method according to claim 28, wherein

said query request includes a semantic temporal event;

said query request includes statistics of the occurrences of a semantic temporal event;

said query request includes statistics of a sports game; and

said query request includes descriptions of actions in a sports game.

15. A system for detecting a semantic temporal event, said system comprising:

a knowledge-based modeling unit for generating multiple-layer models for said semantic temporal event;

a storage mechanism for storing said multiple-layer models;

an observation collection unit for extracting, from at least one data source, temporal observations according to said multiple-layer models for the semantic temporal event; and

a temporal event detection unit for detecting one or more occurences of the semantic temporal event based on said temporal observations and said multiple-layer models.

18. A computer-readable medium encoded with a program for detecting a semantic temporal event, said program comprising:

retrieving multiple-layer models corresponding to said semantic temporal event;

receiving temporal observations that are extracted, from at least one data source, according to said multiple-layer models for the semantic temporal event; and

detecting one or more occurrences of the semantic temporal event based on said temporal observations and said multiple-layer models.

28. A method of querying semantic temporal events, said method comprising:

receiving a query request from a client at a search engine server;

retrieving data requested by said client from a data storage based on event based indexing; and

sending said data retrieved by said trtrieving to said client.