Method and system for defining and recognizing complex events in a video sequence

• US 6,628,835 B1
• Filed: 08/24/1999
• Issued: 09/30/2003
• Est. Priority Date: 08/31/1998
• Status: Expired due to Term

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Olson, Thomas J., Brill, Frank Z.
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/379,986
• Time in Patent Office: 1,498 Days
• Field of Search: 382/218, 382/219, 382/224, 382/225, 382/226, 382/103, 348/142, 348/143, 348/152, 348/153, 348/154, 348/155, 386/69, 340/511
• US Class Current: 382/226
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Given a system which detects simple events, one can define a complex event by constructing a list of sub-events. In order to recognize a complex event, the system keeps a record of the sub-events that have occurred thus far and the objects involved in these sub-events. Whenever the first sub-event in a complex event'"'"'s sequence is recognized, an activation for that complex event is created. The activation contains an indication of the identity of the object involved in the event. The activation also includes an index initialized to one. If a newly detected event matches the next sub-event in any of the currently open complex events, the index for that complex event is incremented. If the index reaches the total number of sub-events in that complex event, the complete complex event is recognized. Thus any desired alarm is generated. Since the complex event that was just recognized may also be a sub-event of another complex event, the activation lists are consulted again to see if the indices of any other complex event activations can be advanced.

Claims

1. A computer implemented method for recognizing complex events in a sequence of video images comprising the steps of:

defining at least one complex event as an ordered list of a predetermined number of events;

recognizing simple events in the sequence of video images;

upon recognition of a simple event

determining if said recognized simple event is a first event in said set ordered list of events of a complex event, and if so activating a complex event recognition process for said corresponding complex event and initializing an index for said complex event recognition process to 1;

determining if said recognized simple event is a next event in said ordered list of events of an activated complex event recognition process, and if so logging said recognized event in said complex event recognition process and incrementing said index of said complex event recognition process; and

determining if all events in an activated complex event recognition process have been detected by determining if said corresponding index equals said predetermined number of events in said corresponding complex event, and if so recognizing said corresponding complex event.

2. The computer implemented method of claim 1, wherein:

said step of defining at least one complex event comprises defining a complex event including another defined complex event;

said method further comprises upon recognition of a complex event

determining if said recognized complex event is a first event in said ordered list of events of another complex event, and if so activating a recognition process for said corresponding complex event and initializing an index for said other complex event recognition process to 1;

determining if said recognized complex event is a next event in said ordered list of events in an activated complex event recognition process, and if so logging said recognized event in said complex event recognition process and incrementing said index of said complex event recognition process; and

determining if all events in an activated complex event recognition process have been detected by determining if said corresponding index equals said predetermined number of events in said corresponding complex event, and if so recognizing said corresponding complex event.

3. The computer implemented method of claim 1, wherein:

said step of defining a complex-event comprises defining a complex event includes at least one negated event;

said method further comprises the steps of upon recognition of an event determining if said recognized event matches a non-negated event corresponding to a next negated event in said ordered list of an activated complex event recognition process,

if so deactivating said complex event recognition process, and

if not determining if said recognized event matches a next non-negated event of an activate complex event recognition process, and if so logging said recognized event and all intervening negated events in said complex event recognition process and incrementing said index for said recognized event and for each intervening negated event.

4. The computer implemented process of claim 3, wherein:

said step of defining a complex event includes an initial negated event; and

said method further comprises determining if said recognized event matches a non-negated event corresponding to said initial negated event of a complex event, and if so disqualifying a corresponding tracked object relative to said corresponding complex event.

5. The computer implemented process of claim 3, wherein:

said step of defining a complex event includes a final negated event; and

said method further comprises upon detection of a final non-negated event in said corresponding complex event, recognizing said complex event if no detection of said negated event occurs within a predetermined period of time.

6. The computer implemented process of claim 3, wherein:

said simple events include an EXIT event wherein an object previously present in said sequence of video images is no longer present;

said step of defining a complex event includes a final negated event; and

said method further comprises upon detection of a final non-negated event in said corresponding complex event, recognizing said complex event if no detection of said negated event occurs before a corresponding object EXITs.