VISUALIZING AND UPDATING CLASSIFICATIONS IN A VIDEO SURVEILLANCE SYSTEM

• US 20110050897A1
• Filed: 08/31/2009
• Published: 03/03/2011
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: 9051147 Canada Incorporated
• Inventors: Cobb, Wesley Kenneth, Blythe, Bobby Ernest, Gottumukkal, Rajkiran Kumar, Friedlander, David Samuel, Saitwal, Kishor Adinath, Xu, Gang, Seow, Ming-Jung
• Granted Patent: US 8,797,405 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for visually conveying classifications derived from pixel-level micro-features extracted from image data. The image data may include an input stream of video frames depicting one or more foreground objects. The classifications represent information learned by a video surveillance system. A request may be received to view a classification. A visual representation of the classification may be generated. A user interface may be configured to display the visual representation of the classification and to allow a user to view and/or modify properties associated with the classification.

Claims

1. A computer-implemented method for a video surveillance system to process a sequence of video frames depicting a scene captured by a video camera, comprising:

receiving a request to view an object classification type assigned to a foreground object depicted in the sequence of video frames, wherein the object classification type classifies the foreground object as being an instance of one of a plurality of foreground object classification types, and wherein the object classification type is assigned to the foreground object based on a plurality of micro-features derived from analyzing pixels depicting the foreground object in the sequence of video frames;

generating a visual representation of the requested object classification type; and

outputting the generated visual representation for display.

2. The computer-implemented method of claim 1, wherein the object classification type corresponds to a cluster in an Adaptive Resonance Theory (ART) network.

3. The computer-implemented method of claim 1, wherein the object classification type corresponds to a node in a self-organizing map (SOM).

4. The computer-implemented method of claim 1, wherein the object classification type is generated by mapping micro-features derived from a plurality of foreground objects to nodes of a self-organizing map (SOM) and wherein an Adaptive Resonance Theory (ART) network clusters resulting nodes in the SOM.

5. The computer-implemented method of claim 1, wherein the micro-features include at least one of hue entropy, magnitude-saturation ratio, orientation angle, pixel area, aspect ratio, groupiness, legged-ness, verticality, animateness, and periodicity of motion of the foreground object in the sequence of video frames.

6. The computer-implemented method of claim 1, further comprising:

receiving user input requesting to modify a metadata attribute of the object classification type assigned to the foreground object; and

modifying the metadata attribute of the object classification, based on the received user input.

7. The computer-implemented method of claim 6, wherein the metadata attribute is a user-specified name for object classification type.

8. The computer-implemented method of claim 6, wherein the metadata attribute specifies to publish an alert message upon detecting an instance of a second foreground object in the scene assigned the object classification type.

9. The computer-implemented method of claim 6, wherein the metadata attribute specifies to not publish an alert message upon detecting an instance of a second foreground object in the scene assigned the object classification type.

10. The computer-implemented method of claim 1, wherein the visual representation of the requested object classification type includes image data from the sequence of video frames depicting one or more instances of the object classification type.

11. A computer-readable storage medium containing a program which, when executed by a video surveillance system, performs an operation to process a sequence of video frames depicting a scene captured by a video camera, the operation comprising:

receiving a request to view an object classification type assigned to a foreground object depicted in the sequence of video frames, wherein the object classification type classifies the foreground object as being an instance of one of a plurality of foreground object classification types, and wherein the object classification type is assigned to the foreground object based on a plurality of micro-features derived from analyzing pixels depicting the foreground object in the sequence of video frames;

generating a visual representation of the requested object classification type; and

outputting the generated visual representation for display.

12. The computer-readable storage medium of claim 11, wherein the object classification type corresponds to a cluster in an Adaptive Resonance Theory (ART) network.

13. The computer-readable storage medium of claim 11, wherein the object classification type corresponds to a node in a self-organizing map (SOM).

14. The computer-readable storage medium of claim 11, wherein the object classification type is generated by mapping micro-features derived from a plurality of foreground objects to nodes of a self-organizing map (SOM) and wherein an Adaptive Resonance Theory (ART) network clusters resulting nodes in the SOM.

15. The computer-readable storage medium of claim 11, wherein the micro-features include at least one of hue entropy, magnitude-saturation ratio, orientation angle, pixel area, aspect ratio, groupiness, legged-ness, verticality, animateness, and periodicity of motion of the foreground object in the sequence of video frames.

16. The computer-readable storage medium of claim 11, wherein the operation further comprises:

receiving user input requesting to modify a metadata attribute of the object classification type assigned to the foreground object; and

modifying the metadata attribute of the object classification, based on the received user input.

17. The computer-readable storage medium of claim 16, wherein the metadata attribute is a user-specified name for object classification type.

18. The computer-readable storage medium of claim 16, wherein the metadata attribute specifies to publish an alert message upon detecting an instance of a second foreground object in the scene assigned the object classification type.

19. The computer-readable storage medium of claim 16, wherein the metadata attribute specifies to not publish an alert message upon detecting an instance of a second foreground object in the scene assigned the object classification type.

20. A video surveillance system, comprising:

a video input source configured to provide a sequence of video frames, each depicting a scene;

a processor; and

a memory containing a program, which when executed by the processor is configured to perform an operation to process the scene depicted in the sequence of video frames, the operation comprising;

receiving a request to view an object classification type assigned to a foreground object depicted in the sequence of video frames, wherein the object classification type classifies the foreground object as being an instance of one of a plurality of foreground object classification types, and wherein the object classification type is assigned to the foreground object based on a plurality of micro-features derived from analyzing pixels depicting the foreground object in the sequence of video frames;

generating a visual representation of the requested object classification type; and

outputting the generated visual representation for display.

21. The video surveillance system of claim 20, wherein the object classification type is generated by mapping micro-features derived from a plurality of foreground objects to nodes of a self-organizing map (SOM) and wherein an Adaptive Resonance Theory (ART) network clusters resulting nodes in the SOM.

22. The video surveillance system of claim 20, wherein the operation further comprises:

receiving user input requesting to modify a metadata attribute of the object classification type assigned to the foreground object; and

modifying the metadata attribute of the object classification, based on the received user input.

23. The video surveillance system of claim 22, wherein the metadata attribute is a user-specified name for object classification type.

24. The video surveillance system of claim 22, wherein the metadata attribute specifies to publish an alert message upon detecting an instance of a second foreground object in the scene assigned the object classification type.

25. The video surveillance system of claim 22, wherein the metadata attribute specifies to not publish an alert message upon detecting an instance of a second foreground object in the scene assigned the object classification type.