Visualizing and updating learned trajectories in video surveillance systems

• US 8,280,153 B2
• Filed: 08/18/2009
• Issued: 10/02/2012
• Est. Priority Date: 08/18/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Blythe, Bobby Ernest, Friedlander, David Samuel, Gottumukkal, Rajkiran Kumar, Seow, Ming-Jung, Xu, Gang
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US12/543,242
• Publication Number:
  US 20110044498A1
• Time in Patent Office: 1,141 Days
• Field of Search: 382/103, 382/155
• US Class Current: 382/155
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

Techniques are disclosed for visually conveying a trajectory map. The trajectory map provides users with a visualization of data observed by a machine-learning engine of a behavior recognition system. Further, the visualization may provide an interface used to guide system behavior. For example, the interface may be used to specify that the behavior recognition system should alert (or not alert) when a particular trajectory is observed to occur.

Claims

1. A computer-implemented method of generating a display of information learned by a video surveillance system, comprising:

receiving a request to view a trajectory map for a scene depicted in a sequence of video frames captured by a video camera;

retrieving a background image associated with the scene;

retrieving one or more trajectories associated with one or more foreground objects depicted in the sequence of video frames, wherein each trajectory plots a path traversed by a respective foreground object in moving through the scene; and

superimposing a visual representation of each retrieved trajectory over the background image at a location corresponding to the path traversed by the respective foreground object in moving through the scene.

2. The computer-implemented method of claim 1, wherein a support vector machine classifies each retrieved trajectory as being anomalous or not anomalous, relative to trajectories of a plurality of foreground objects, and wherein the visual representation of identifies retrieved trajectories classified as being anomalous.

3. The computer-implemented method of claim 1, wherein the background image specifies a pixel value for each pixel expected to be observed in a frame of video captured by the video camera when scene background is visible to the video camera in a frame of video.

4. The computer-implemented method of claim 1, wherein the visual representation of each retrieved trajectory identifies pixels in the background image used to plot the path of the corresponding foreground object in moving through the scene.

5. The computer-implemented method of claim 1, wherein the identified pixels are determined relative to a geometric center of the foreground object, as depicted in each of a sequence of frames.

6. The computer-implemented method of claim 1, wherein at least one retrieved trajectory is a composite generated from multiple retrieved trajectories observed at the scene.

7. The computer-implemented method of claim 1, further comprising:

receiving, as user input, metadata to associate with a first one of the retrieved trajectories, wherein the metadata is selected from at least;

(i) a label to assign to occurrences the first trajectory observed in the sequence of video frames;

(ii) an indication to generate an alert message each time the first trajectory subsequently observed; and

(iii) an indication to not generate an alert message each time the first trajectory subsequently observed.

8. The computer-implemented method of claim 1, further comprising, receiving an indication of an object classification type, wherein the retrieved trajectories are associated with foreground objects classified as being an instance of the object classification type.

9. A non-transitory computer-readable medium containing a program which, when executed, performs an operation for generating a display of information learned by a video surveillance system, the operation comprising:

receiving a request to view a trajectory map for a scene depicted in a sequence of video frames captured by a video camera;

retrieving a background image associated with the scene;

retrieving one or more trajectories associated with one or more foreground objects depicted in the sequence of video frames, wherein each trajectory plots a path traversed by a respective foreground object in moving through the scene; and

superimposing a visual representation of each retrieved trajectory over the background image at a location corresponding to the path traversed by the respective foreground object in moving through the scene.

10. The non-transitory computer-readable medium of claim 9, wherein a support vector machine classifies each retrieved trajectory as being anomalous or not anomalous, relative to trajectories of a plurality of foreground objects, and wherein the visual representation of identifies retrieved trajectories classified as being anomalous.

11. The non-transitory computer-readable medium of claim 9, wherein the background image specifies a pixel value for each pixel expected to be observed in a frame of video captured by the video camera when scene background is visible to the video camera in a frame of video.

12. The non-transitory computer-readable medium of claim 9, wherein the visual representation of each retrieved trajectory identifies pixels in the background image used to plot the path of the corresponding foreground object in moving through the scene.

13. The non-transitory computer-readable medium of claim 9, wherein the identified pixels are determined relative to a geometric center of the foreground object, as depicted in each of a sequence of frames.

14. The non-transitory computer-readable medium of claim 9, wherein at least one retrieved trajectory is a composite generated from multiple retrieved trajectories observed at the scene.

15. The non-transitory computer-readable medium of claim 9, wherein the operation further comprises:

receiving, as user input, metadata to associate with a first one of the retrieved trajectories, wherein the metadata is selected from at least;

(i) a label to assign to occurrences the first trajectory observed in the sequence of video frames;

(ii) an indication to generate an alert message each time the first trajectory subsequently observed; and

(iii) an indication to not generate an alert message each time the first trajectory subsequently observed.

16. The non-transitory computer-readable medium of claim 9, wherein the operation further comprises, receiving an indication of an object classification type, wherein the retrieved trajectories are associated with foreground objects classified as being an instance of the object classification type.

17. A system, comprising:

a video camera;

one or more computer processors; and

a memory containing a program, which, when executed by the one or more computer processors, performs an operation for generating a display of information, the operation comprising;

receiving a request to view a trajectory map for a scene depicted in a sequence of video frames captured by the video camera,

retrieving a background image associated with the scene,

retrieving one or more trajectories associated with one or more foreground objects depicted in the sequence of video frames, wherein each trajectory plots a path traversed by a respective foreground object in moving through the scene, and

superimposing a visual representation of each retrieved trajectory over the background image at a location corresponding to the path traversed by the respective foreground object in moving through the scene.

18. The system of claim 17, wherein a support vector machine classifies each retrieved trajectory as being anomalous or not anomalous, relative to trajectories of a plurality of foreground objects, and wherein the visual representation of identifies retrieved trajectories classified as being anomalous.

19. The system of claim 17, wherein the background image specifies a pixel value for each pixel expected to be observed in a frame of video captured by the video camera when scene background is visible to the video camera in a frame of video.

20. The system of claim 17, wherein the visual representation of each retrieved trajectory identifies pixels in the background image used to plot the path of the corresponding foreground object in moving through the scene.

21. The system of claim 17, wherein the identified pixels are determined relative to a geometric center of the foreground object, as depicted in each of a sequence of frames.

22. The system of claim 17, wherein at least one retrieved trajectory is a composite generated from multiple retrieved trajectories observed at the scene.

23. The system of claim 17, wherein the operation further comprises:

receiving, as user input, metadata to associate with a first one of the retrieved trajectories, wherein the metadata is selected from at least;

(i) a label to assign to occurrences the first trajectory observed in the sequence of video frames;

(ii) an indication to generate an alert message each time the first trajectory subsequently observed; and

(iii) an indication to not generate an alert message each time the first trajectory subsequently observed.

24. The system of claim 17, wherein the operation further comprises, receiving an indication of an object classification type, wherein the retrieved trajectories are associated with foreground objects classified as being an instance of the object classification type.