Visualizing and updating learned event maps in surveillance systems

• US 8,625,884 B2
• Filed: 08/18/2009
• Issued: 01/07/2014
• Est. Priority Date: 08/18/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Blythe, Bobby Ernest, Gottumukkal, Rajkiran Kumar, Seow, Ming-Jung
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US12/543,204
• Publication Number:
  US 20110044533A1
• Time in Patent Office: 1,603 Days
• Field of Search: 382/155, 382/156, 382/180, 348/148, 348/E07.085
• US Class Current: 382/155
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

Techniques are disclosed for visually conveying an event map. The event map may represent information learned by a surveillance system. A request may be received to view the event map for a specified scene. The event map may be generated, including a background model of the specified scene and at least one cluster providing a statistical distribution of an event in the specified scene. Each statistical distribution may be derived from data streams generated from a sequence of video frames depicting the specified scene captured by a video camera. Each event may be observed to occur at a location in the specified scene corresponding to a location of the respective cluster in the event map. The event map may be configured to allow a user to view and/or modify properties associated with each cluster. For example, the user may label a cluster and set events matching the cluster to always (or never) generate an alert.

Claims

1. A computer-implemented method, comprising:

receiving a request to view an event map of a specified event type for a scene depicted in a sequence of video frames;

retrieving a background image associated with the scene;

retrieving an adaptive resonance theory (ART) network modeling the specified event type, wherein the ART network is generated from the sequence of video frames depicting the scene captured by a video camera, and wherein a location of each cluster in the ART network models a region of the scene where one or more events of the specified type has been to observed to occur;

generating a visual representation of each cluster; and

superimposing each visual representation over the background image of the scene.

2. The computer-implemented method of claim 1, wherein the visual representation of each cluster is generated at least in part from values of properties of the respective cluster, and wherein at least one of shape, size, transparency, orientation, fill color, border thickness, border color, pattern, label, and marking of the visual representation is determined from values of properties of the respective cluster.

3. The computer-implemented method of claim 1, further comprising:

receiving user input specifying a filter condition for selecting one or more clusters from the ART network to include in the event map; and

excluding clusters from the event map consistent with the filter condition.

4. The computer-implemented method of claim 1, further comprising:

receiving user input modifying a property of a specified cluster, wherein the property is selected from at least;

(i) a name for the specified cluster;

(ii) whether an alert message is to be published upon detecting, from data streams generated from the sequence of video frames, an occurrence of an event mapping to the specified cluster; and

(iii) whether an alert message is not to be published even upon detecting, from the data streams, an occurrence of an event matching the specified cluster; and

modifying the property of the specified cluster, based on the received user input.

5. The computer-implemented method of claim 4, wherein the data streams include:

(i) a stream of context events generated by a computer vision engine, wherein each context event provides kinematic data related to a foreground object observed by the computer vision engine in the sequence of video frames; and

(ii) a stream of primitive events generated by a machine learning engine, and wherein each primitive event provides a semantic description of a group of one or more context events.

6. The computer-implemented method of claim 5, wherein one of the context events provides a classification of what is depicted by a foreground object detected in the scene by the computer vision engine, and wherein the classification classifies the foreground object as depicting one of a person, a vehicle, or an unknown, and an other class of foreground object.

7. The computer-implemented method of claim 5, wherein the kinematic data includes at least one of a coordinate position in a frame of video where an event is observed to occur, and wherein the event is one of an appearance of a foreground object, a disappearance of a foreground object, a height of a foreground object, a width of a foreground object, a velocity in a horizontal dimension of a foreground object, a velocity in a vertical dimension of a foreground object, a rate of acceleration in a horizontal dimension of a foreground object, and a rate of acceleration in a vertical dimension of a foreground object.

8. A computer-readable storage medium containing a program which, when executed, performs an operation comprising:

receiving a request to view an event map of a specified event type for a scene depicted in a sequence of video frames;

retrieving a background image associated with the scene;

retrieving an adaptive resonance theory (ART) network modeling the specified event type, wherein the ART network is generated from the sequence of video frames depicting the scene captured by a video camera, and wherein a location of each cluster in the ART network models a region of the scene where one or more events of the specified type has been to observed to occur;

generating a visual representation of each cluster; and

superimposing each visual representation over the background image of the scene.

9. The computer-readable storage medium of claim 8, wherein the visual representation of each cluster is generated at least in part from values of properties of the respective cluster, and wherein at least one of shape, size, transparency, orientation, fill color, border thickness, border color, pattern, label, and marking of the visual representation is determined from values of properties of the respective cluster.

10. The computer-readable storage medium of claim 8, wherein the operation further comprises:

receiving user input specifying a filter condition for selecting one or more clusters from the ART network to include in the event map; and

excluding clusters from the event map consistent with the filter condition.

11. The computer-readable storage medium of claim 8, wherein the operation further comprises:

receiving user input modifying a property of a specified cluster, wherein the property is selected from at least;

(i) a name for the specified cluster;

(ii) whether an alert message is to be published upon detecting, from data streams generated from the sequence of video frames, an occurrence of an event mapping to the specified cluster; and

(iii) whether an alert message is not to be published even upon detecting, from the data streams, an occurrence of an event matching the specified cluster; and

modifying the property of the specified cluster, based on the received user input.

12. The computer-readable storage medium of claim 11, wherein the data streams include:

(i) a stream of context events generated by a computer vision engine, wherein each context event provides kinematic data related to a foreground object observed by the computer vision engine in the sequence of video frames; and

(ii) a stream of primitive events generated by a machine learning engine, and wherein each primitive event provides a semantic description of a group of one or more context events.

13. The computer-readable storage medium of claim 12, wherein one of the context events provides a classification of what is depicted by a foreground object detected in the scene by the computer vision engine, and wherein the classification classifies the foreground object as depicting one of a person, a vehicle, or an unknown, and an other class of foreground object.

14. The computer-readable storage medium of claim 12, wherein the kinematic data includes at least one of a coordinate position in a frame of video where an event is observed to occur, and wherein the event is one of an appearance of a foreground object, a disappearance of a foreground object, a height of a foreground object, a width of a foreground object, a velocity in a horizontal dimension of a foreground object, a velocity in a vertical dimension of a foreground object, a rate of acceleration in a horizontal dimension of a foreground object, and a rate of acceleration in a vertical dimension of a foreground object.

15. A system, comprising:

one or more computer processors; and

a memory containing a program, which when executed by the one or more computer processors is configured to perform an operation comprising;

receiving a request to view an event map of a specified event type for a scene depicted in a sequence of video frames;

retrieving a background image associated with the scene;

retrieving an adaptive resonance theory (ART) network modeling the specified event type, wherein the ART network is generated from the sequence of video frames depicting the scene captured by a video camera, and wherein a location of each cluster in the ART network models a region of the scene where one or more events of the specified type has been to observed to occur;

generating a visual representation of each cluster; and

superimposing each visual representation over the background image of the scene.

16. The system of claim 15, wherein the visual representation of each cluster is generated at least in part from values of properties of the respective cluster, and wherein at least one of shape, size, transparency, orientation, fill color, border thickness, border color, pattern, label, and marking of the visual representation is determined from values of properties of the respective cluster.

17. The system of claim 15, wherein the operation further comprises:

receiving user input specifying a filter condition for selecting one or more clusters from the ART network to include in the event map; and

excluding clusters from the event map consistent with the filter condition.

18. The system of claim 15, wherein the operation further comprises:

receiving user input modifying a property of a specified cluster, wherein the property is selected from at least;

(i) a name for the specified cluster;

(ii) whether an alert message is to be published upon detecting, from data streams generated from the sequence of video frames, an occurrence of an event mapping to the specified cluster; and

(iii) whether an alert message is not to be published even upon detecting, from the data streams, an occurrence of an event matching the specified cluster; and

modifying the property of the specified cluster, based on the received user input.

19. The system of claim 18, wherein the data streams include:

(i) a stream of context events generated by a computer vision engine, wherein each context event provides kinematic data related to a foreground object observed by the computer vision engine in the sequence of video frames; and

(ii) a stream of primitive events generated by a machine learning engine, and wherein each primitive event provides a semantic description of a group of one or more context events.

20. The system of claim 19, wherein one of the context events provides a classification of what is depicted by a foreground object detected in the scene by the computer vision engine, and wherein the classification classifies the foreground object as depicting one of a person, a vehicle, or an unknown, and an other class of foreground object.

21. The system of claim 19, wherein the kinematic data includes at least one of a coordinate position in a frame of video where an event is observed to occur, and wherein the event is one of an appearance of a foreground object, a disappearance of a foreground object, a height of a foreground object, a width of a foreground object, a velocity in a horizontal dimension of a foreground object, a velocity in a vertical dimension of a foreground object, a rate of acceleration in a horizontal dimension of a foreground object, and a rate of acceleration in a vertical dimension of a foreground object.