Method and system for detecting semantic events

• US 6,678,635 B2
• Filed: 01/23/2001
• Issued: 01/13/2004
• Est. Priority Date: 01/23/2001
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Qian, Richard J., Tovinkere, Vasanth R., Epshteyn, Eugene
• Primary Examiner(s): WACHSMAN, HAL D
• Application Number:
  US09/766,594
• Publication Number:
  US 20020099518A1
• Time in Patent Office: 1,085 Days
• Field of Search: 702/176-181, 702/183, 702/186-188, 702/FOR.103, 702/FOR.104, 702/FOR.114, 702/FOR.134, 702/FOR.135, 702/FOR.139, 702/FOR.154, 702/FOR.170, 706/15, 706/20, 706/21, 706/8, 706/1, 706/26, 706/31, 706/45-48, 706/50, 706/52, 706/55, 706/58, 706/900, 706/16, 706/23, 706/25, 703/2, 703/3, 707/3-6, 707/500.1, 707/501.1, 386/68, 386/69, 386/111, 348/157, 348/155, 348/135, 725/45, 725/46, 343/719-721, 382/103, 382/155, 382/156, 382/159, 704/232, 704/255, 704/259, 273/259, 273/277, 434/251, 434/257
• US Class Current: 702/179
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A method and system is provided for detecting occurrences of semantic temporal events based on observations extracted from input data and event models. The input data is fed into the system from some data source. Based on specified event to be detected, multiple-layer models corresponding to the event are retrieved. The models are used to determine the types of temporal observations to be extracted from the input data. The extracted temporal observations are then used, in combination with the multiple-layer models of the event, to detect the occurrences of the event.

Claims

1. A method for detecting a semantic temporal event, said method comprising:

retrieving multiple-layer models corresponding to said semantic temporal event;

receiving temporal observations that are extracted, from at least one data source, according to said multiple-layer models for the semantic temporal event;

detecting one or more occurrences of the semantic temporal event based on said temporal observations and said multiple-layer models by supplying said temporal observations to said multiple-layer models;

characterizing said one or more occurrences of the semantic temporal event, detected by said detecting, to produce a characterization;

storing said characterization;

performing temporal event prediction based on said characterization;

revising said multiple-layer models for said semantic temporal event based on said characterization; and

simulating parts of said semantic temporal event according to said characterization.

2. The method according to claim 1, wherein said semantic temporal event includes a sports event.

3. The method according to claim 2, wherein said sports event includes a soccer game.

4. The method according to claim 1, wherein said multiple-layer models include a high level domain-specific knowledge model and a dynamic hierarchical event model.

5. The method according to claim 4, wherein said high level domain-specific knowledge model includes rules of a sports game.

6. The method according to claim 4, wherein said dynamic hierarchical event model includes a hierarchical decision tree.

7. The method according to claim 4, wherein said dynamic hierarchical event model includes an entity-relationship-diagram.

8. The method according to claim 1, wherein said at least one data source includes at least one data acquisition device including a camera, a microwave sensor, a sound recorder, and an input data stream selected from the group consisting of video, audio, text, and temporal features.

9. The method according to claim 8, wherein said temporal features include tracking position data.

10. The method according to claim 1, wherein said detecting comprises:

performing semantic temporal event detection using a plurality of detection methods, each of said plurality of detection methods producing a detection result; and

combining said detection results with each other to produce a final detection.

11. The method according to claim 10, wherein said plurality of detection methods includes dynamic Bayesian networks, rule based expert systems, decision trees, Hidden Markov Models, neural networks, or fuzzy logic.

12. The method according to claim 1, wherein said characterization includes:

a set of statistics computed from said one or more occurrences;

or

a set of descriptions, each of which describes an action happening in an occurrence of the semantic temporal event.

13. A computer-readable medium for programming a computer to detect a semantic temporal event, comprising instructions for:

retrieving multiple-layer models corresponding to said semantic temporal event;

receiving temporal observations that are extracted, from at least one data source, according to said multiple-layer models for the semantic temporal event;

detecting one or more occurrences of the semantic temporal event based on said temporal observations and said multiple-layer models;

characterizing said one or more occurrences of the semantic temporal event, detected by said detecting, to produce a characterization;

storing said characterization;

performing temporal event prediction based on said characterization;

revising said multiple-layer models based on said characterization; and

simulating parts of said semantic temporal event according to said characterization.

14. The computer-readable medium according to claim 13, wherein said semantic temporal event includes a sports temporal event.

15. The computer-readable medium according to claim 13, wherein said semantic temporal event includes a soccer game.

16. The computer-readable medium according to claim 13, wherein said multiple-layer models include a high level domain-specific knowledge model and a dynamic hierarchical event model.

17. The computer-readable medium according to claim 16, wherein said high level domain-specific knowledge model includes rules of a sports game.

18. The computer-readable medium according to claim 13, wherein said detecting comprises:

performing semantic temporal event detection using a plurality of detection methods, each of said plurality of detection methods producing a detection result; and

combining said detection results with each other to produce a detection.

19. The computer-readable medium according to claim 18, wherein said plurality of detection methods includes dynamic Bayesian networks, rule based expert systems, decision trees, Hidden Markov Models, neural networks, or fuzzy logic.

20. The computer-readable medium according to claim 13, wherein said characterization includes:

a set of statistics computed from said one or more occurrences;

or

a set of descriptions, each of which describes an action happening in an occurrence of the semantic temporal event.