Visualizing and updating sequences and segments in a video surveillance system

• US 8,493,409 B2
• Filed: 08/18/2009
• Issued: 07/23/2013
• Est. Priority Date: 08/18/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Blythe, Bobby Ernest, Friedlander, David Samuel, Gottumukkal, Rajkiran Kumar, Saitwal, Kishor Adinath
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US12/543,351
• Publication Number:
  US 20110043536A1
• Time in Patent Office: 1,435 Days
• Field of Search: 345/629, 345/619, 345/419, 345/581, 345/420, 345/633
• US Class Current: 345/629
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Techniques are disclosed for visually conveying a sequence storing an ordered string of symbols generated from kinematic data derived from analyzing an input stream of video frames depicting one or more foreground objects. The sequence may represent information learned by a video surveillance system. A request may be received to view the sequence or a segment partitioned form the sequence. A visual representation of the segment may be generated and superimposed over a background image associated with the scene. A user interface may be configured to display the visual representation of the sequence or segment and to allow a user to view and/or modify properties associated with the sequence or segment.

Claims

1. A computer-implemented method, comprising:

receiving a request to view a sequence storing an ordered string of symbols, wherein the sequence is generated from kinematic data derived from analyzing an input stream of video frames depicting one or more foreground objects and wherein the sequence is partitioned into one or more segments using by a collection of voting experts configured to vote positions within the sequence at which to partition the sequence into the one or more segments;

retrieving a background image associated with a scene depicted in the input stream of video frames;

generating a visual representation of the requested sequence; and

superimposing the visual representation of the requested sequence over the background image, wherein the position of each symbol of the ordered string of symbols in the sequence is superimposed at a position in the background image corresponding to a spatial location in the scene where the kinematic data associated with the respective symbol was obtained.

2. The computer implemented method of claim 1, wherein the request to view the sequence is a request to view one or more of the partitioned segments.

3. The computer-implemented method of claim 1, wherein each symbol in the string of symbols corresponds to a cluster in an adaptive resonance theory (ART) network.

4. The computer-implemented method of claim 3, wherein the string of symbols is generated by mapping the kinematic data for a first foreground object detected in the input stream of video frames to nodes of a self-organizing map (SOM) and wherein the ART network clusters resulting nodes in the SOM.

5. The computer-implemented method of claim 1, wherein the background image specifies a pixel value expected to be observed by the video camera for each pixel in a frame of video when scene background is visible to the video camera.

6. The computer-implemented method of claim 1, further comprising:

receiving, as user input, a property to associate with the requested sequence, wherein the property is selected from at least;

(i) a label to assign to the sequence;

(ii) an indication to generate an alert message each time the sequence is subsequently observed; and

(iii) an indication to not generate an alert message any time the sequence is subsequently observed.

7. A computer-readable storage medium containing a program, which when executed on a processor, performs an operation, comprising:

receiving a request to view a sequence storing an ordered string of symbols, wherein the sequence is generated from kinematic data derived from analyzing an input stream of video frames depicting one or more foreground objects and wherein the sequence is partitioned into one or more segments using by a collection of voting experts configured to vote positions within the sequence at which to partition the sequence into the one or more segments;

retrieving a background image associated with a scene depicted in the input stream of video frames;

generating a visual representation of the requested sequence; and

superimposing the visual representation of the requested sequence over the background image, wherein the position of each symbol of the ordered string of symbols in the sequence is superimposed at a position in the background image corresponding to a spatial location in the scene where the kinematic data associated with the respective symbol was obtained.

8. The computer-readable storage medium of claim 7, wherein the request to view the sequence is a request to view one or more of the partitioned segments.

9. The computer-readable storage medium of claim 7, wherein each symbol in the string of symbols corresponds to a cluster in an adaptive resonance theory (ART) network.

10. The computer-readable storage medium of claim 9, wherein the string of symbols is generated by mapping the kinematic data for a first foreground object detected in the input stream of video frames to nodes of a self-organizing map (SOM) and wherein the ART network clusters resulting nodes in the SOM.

11. The computer-readable storage medium of claim 7, wherein the background image specifies a pixel value expected to be observed by the video camera for each pixel in a frame of video when scene background is visible to the video camera.

12. The computer-readable storage medium of claim 7, wherein the operation further comprises:

receiving, as user input, a property to associate with the requested sequence, wherein the property is selected from at least;

(i) a label to assign to the sequence;

(ii) an indication to generate an alert message each time the sequence is subsequently observed; and

(iii) an indication to not generate an alert message any time the sequence is subsequently observed.

13. A system, comprising:

a video input source configured to provide an input stream of video frames, each depicting a scene;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation, comprising;

receiving a request to view a sequence storing an ordered string of symbols, wherein the sequence is generated from kinematic data derived from analyzing one or more foreground objects depicted in the input stream of video frames and wherein the sequence is partitioned into one or more segments using by a collection of voting experts configured to vote positions within the sequence at which to partition the sequence into the one or more segments,

retrieving a background image associated with a scene depicted in the input stream of video frames,

generating a visual representation of the requested sequence, and

superimposing the visual representation of the requested sequence over the background image, wherein the position of each symbol of the ordered string of symbols in the sequence is superimposed at a position in the background image corresponding to a spatial location in the scene where the kinematic data associated with the respective symbol was obtained.

14. The system of claim 13, wherein the request to view the sequence is a request to view one or more of the partitioned segments.

15. The system of claim 13, wherein each symbol in the string of symbols corresponds to a cluster in an adaptive resonance theory (ART) network.

16. The system of claim 13, wherein the string of symbols is generated by mapping the kinematic data for a first foreground object detected in the input stream of video frames to nodes of a self-organizing map (SOM) and wherein the ART network clusters resulting nodes in the SOM.

17. The system of claim 13, wherein the background image specifies a pixel value expected to be observed by the video camera for each pixel in a frame of video when scene background is visible to the video camera.

18. The system of claim 13, wherein the operation further comprises:

receiving, as user input, a property to associate with the requested sequence, wherein the property is selected from at least;

(i) a label to assign to the sequence;

(ii) an indication to generate an alert message each time the sequence is subsequently observed; and

(iii) an indication to not generate an alert message any time the sequence is subsequently observed.