Visualizing and updating long-term memory percepts in a video surveillance system

• US 8,786,702 B2
• Filed: 08/31/2009
• Issued: 07/22/2014
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Blythe, Bobby Ernest, Gottumukkal, Rajkiran Kumar, Seow, Ming-Jung
• Primary Examiner(s): Dollinger, Tonia L
• Assistant Examiner(s): COONEY, ADAM A
• Application Number:
  US12/551,303
• Publication Number:
  US 20110050896A1
• Time in Patent Office: 1,786 Days
• Field of Search: 348/143, 348/135, 348/142, 348/146, 348/149, 348/159
• US Class Current: 348/143
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Techniques are disclosed for visually conveying a percept. The percept may represent information learned by a video surveillance system. A request may be received to view a percept for a specified scene. The percept may have been derived from data streams generated from a sequence of video frames depicting the specified scene captured by a video camera. A visual representation of the percept may be generated. A user interface may be configured to display the visual representation of the percept and to allow a user to view and/or modify metadata attributes with the percept. For example, the user may label a percept and set events matching the percept to always (or never) result in alert being generated for users of the video surveillance system.

Claims

1. A computer-implemented method for a video surveillance system to process a sequence of video frames depicting a scene captured by a video camera, comprising:

receiving a request to view a visual representation of a percept encoded in a long-term memory of a machine-learning engine, wherein the percept encodes a pattern of behavior learned by the machine-learning engine from analyzing data streams generated from the sequence of video frames;

retrieving the requested percept from the long-term memory of the machine-learning engine, wherein the long-term memory stores a plurality of percepts; and

generating a visual representation of the requested percept, wherein the visual representation presents a directed graph representing the pattern of behavior encoded by the requested percept; and

receiving user input for a metadata attribute of the requested percept, wherein the metadata attribute specifies to publish an alert message upon detecting, from the data streams, an observation of the learned pattern of behavior corresponding to the requested percept.

2. The computer-implemented method of claim 1, wherein the directed graph includes one or more nodes and links between the nodes, wherein each node represents one or more primitive events observed by the video surveillance system in the sequence of video frames and wherein each link represents a relationship between primitive events in the pattern of behavior.

3. The computer-implemented method of claim 2, wherein generating the visual representation comprises:

retrieving a semantic label associated with each node of the directed graph; and

generating, from the retrieved semantic labels, a clause describing the primitive events and the links between primitive events.

4. The computer-implemented method of claim 1, wherein the metadata attribute includes a user-specified name for the requested percept.

6. The computer-implemented method of claim 1, further comprising:

retrieving, from the long-term memory, a list of the plurality of percepts for the scene; and

displaying the retrieved list of percepts, wherein the requested percept is selected from the displayed list of percepts.

5. A computer-implemented method for a video surveillance system to process a sequence of video frames depicting a scene captured by a video camera, comprising:

receiving a request to view a visual representation of a percept encoded in a long-term memory of a machine-learning engine, wherein the percept encodes a pattern of behavior learned by the machine-learning engine from analyzing data streams generated from the sequence of video frames;

retrieving the requested percept from the long-term memory of the machine-learning engine, wherein the long-term memory stores a plurality of percepts;

generating a visual representation of the requested percept, wherein the visual representation presents a directed graph representing the pattern of behavior encoded by the requested percept; and

receiving user input for a metadata attribute of the requested percept, wherein the metadata attribute specifies to not publish an alert message upon detecting, from the data streams, an observation of the learned pattern of behavior corresponding to the requested percept.

7. A non-transitory computer-readable storage medium containing a program which, when executed by a video surveillance system, performs an operation to process a sequence of video frames depicting a scene captured by a video camera, the operation comprising:

receiving a request to view a visual representation of a percept encoded in a long-term memory of a machine-learning engine, wherein the percept encodes a pattern of behavior learned by the machine-learning engine from analyzing data streams generated from the sequence of video frames;

retrieving the requested percept from the long-term memory of the machine-learning engine, wherein the long-term memory stores a plurality of percepts; and

generating a visual representation of the requested percept, wherein the visual representation presents a directed graph representing the pattern of behavior encoded by the requested percept; and

receiving user input for a metadata attribute of the requested percept, wherein the metadata attribute specifies to publish an alert message upon detecting, from the data streams, an observation of the learned pattern of behavior corresponding to the requested percept.

8. The computer-readable storage medium of claim 7, wherein the directed graph includes one or more nodes and links between the nodes, wherein each node represents one or more primitive events observed by the video surveillance system in the sequence of video frames and wherein each link represents a relationship between primitive events in the pattern of behavior.

9. The computer-readable storage medium of claim 8, wherein generating the visual representation comprises:

retrieving a semantic label associated with each node of the directed graph; and

generating, from the retrieved semantic labels, a clause describing the primitive events and the links between primitive events.

10. The computer-readable storage medium of claim 7, wherein the metadata attribute includes is a user-specified name for the requested percept.

11. A non-transitory computer-readable storage medium containing a program which, when executed by a video surveillance system, performs an operation to process a sequence of video frames depicting a scene captured by a video camera, the operation comprising:

receiving a request to view a visual representation of a percept encoded in a long-term memory of a machine-learning engine, wherein the percept encodes a pattern of behavior learned by the machine-learning engine from analyzing data streams generated from the sequence of video frames;

retrieving the requested percept from the long-term memory of the machine-learning engine, wherein the long-term memory stores a plurality of percepts;

generating a visual representation of the requested percept, wherein the visual representation presents a directed graph representing the pattern of behavior encoded by the requested percept; and

receiving user input for a metadata attribute of the requested percept, wherein the metadata attribute specifies to not publish an alert message upon detecting, from the data streams, an observation of the learned pattern of behavior corresponding to the requested percept.

12. A video surveillance system, comprising:

a video input source configured to provide a sequence of video frames, each depicting a scene;

a processor; and

a memory containing a program, which when executed by the processor is configured to perform an operation to process the scene depicted in the sequence of video frames, the operation comprising;

receiving a request to view a visual representation of a percept encoded in a long-term memory of a machine-learning engine, wherein the percept encodes a pattern of behavior learned by the machine-learning engine from analyzing data streams generated from the sequence of video frames,

retrieving the requested percept from the long-term memory of the machine-learning engine, wherein the long-term memory stores a plurality of percepts, and

generating a visual representation of the requested percept, wherein the visual representation presents a directed graph representing the pattern of behavior encoded by the requested percept, and

receiving user input for a metadata attribute of the requested percept, wherein the metadata attribute specifies to publish an alert message upon detecting, from the data streams, an observation of the learned pattern of behavior corresponding to the requested percept.

13. The system of claim 12, wherein the directed graph includes one or more nodes and links between the nodes, wherein each node represents one or more primitive events observed by the video surveillance system in the sequence of video frames and wherein each link represents a relationship between primitive events in the pattern of behavior.

14. The system of claim 13, wherein generating the visual representation comprises:

retrieving a semantic label associated with each node of the directed graph; and

generating, from the retrieved semantic labels, a clause describing the primitive events and the links between primitive events.

15. The system of claim 13, wherein the metadata attribute includes a user-specified name for the requested percept.

16. A video surveillance system, comprising:

a video input source configured to provide a sequence of video frames, each depicting a scene;

a processor; and

a memory containing a program, which when executed by the processor is configured to perform an operation to process the scene depicted in the sequence of video frames, the operation comprising;

receiving a request to view a visual representation of a percept encoded in a long-term memory of a machine-learning engine, wherein the percept encodes a pattern of behavior learned by the machine-learning engine from analyzing data streams generated from the sequence of video frames,

retrieving the requested percept from the long-term memory of the machine-learning engine, wherein the long-term memory stores a plurality of percepts,

generating a visual representation of the requested percept, wherein the visual representation presents a directed graph representing the pattern of behavior encoded by the requested percept, and

receiving user input for a metadata attribute of the requested percept, wherein the metadata attribute specifies to not publish an alert message upon detecting, from the data streams, an observation of the learned pattern of behavior corresponding to the requested percept.