Threat-detection in a distributed multi-camera surveillance system

• US 8,760,519 B2
• Filed: 02/16/2007
• Issued: 06/24/2014
• Est. Priority Date: 02/16/2007
• Status: Expired due to Fees

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Ozdemir, Hasan Timucin, Lee, Kuo Chu
• Primary Examiner(s): GREENE, JOSEPH L
• Application Number:
  US11/675,849
• Publication Number:
  US 20080198231A1
• Time in Patent Office: 2,685 Days
• Field of Search: 348152-155, 348/159
• US Class Current: 348/159
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 25/009 Signalling of the alarm condition to a substation whose identity is signalled to a central station, e.g. relaying alarm signals in order to extend communication range
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A method is provided for detecting a threat in a distributed multi-camera surveillance system. The method includes: monitoring movement of an object in a field of view of a first camera using software installed at the first camera; detecting a suspicious object at the first camera when movement of the object does not conform with a motion flow model residing at the first camera; sending a tracking request from the first camera to a second camera upon detecting the suspicious object at the first camera; monitoring movement of the object in a field of view of the second camera using software installed at the second camera; assigning threat scores at the second camera when the movement of the object does not conform with a motion flow model residing at the second camera; and generating an alarm based in part on the threat scores detected at the first camera and the second camera.

Claims

1. A distributed multi-camera surveillance system, comprising:

a plurality of cameras including;

a first camera having a field of view and an associated data store for storing a trained statistical motion flow model for objects moving in the field of view of the first camera, the model based on observed motion vectors within the first camera'"'"'s field of view in combination with feedback data compiled and communicated peer-to-peer to the camera by another camera in response to a tracking request message issued by the first camera, the tracking request message communicating time stamp and flow identifier information associated with a trained statistical motion flow, and the feedback data received by said another camera communicating matching time stamp and corresponding flow identifier information that is used by the first camera to dynamically learn a topological relationship between the first camera and said another camera, wherein the motion flow model is a building algorithm embodied in software installed at each of the individual cameras of the plurality of cameras for constructing the motion flow model for their own independent fields of view and wherein the dynamic learning of the topological relationships between the cameras includes each individual camera dynamically updating its statistical model, in real-time, upon the receiving of the dynamically transmitted tracking requests transmitted by the other individual cameras, such that the individual cameras are able to dynamically and individually adapt to their individual orientations and positions;

the system further including;

the first camera operable to detect a threat when movement of an object in its field of view does not conform to the motion flow model and transmit a tracking request for the object over a network to a second camera; and

the second camera having a field of view and an associated data store for storing a motion flow model for objects moving in the field of view of the second camera, the second camera operable to detect a threat when movement of an object in its field of view does not conform to the motion flow model and generate an alarm based on the threat detected at the first and second cameras.

2. The surveillance system of claim 1 further comprises a tracking algorithm embodied in software installed at the first camera for monitoring movement of objects in its field of view.

3. The surveillance system of claim 1 further comprises a camera association algorithm embodied in software installed at the first camera for learning associations with other cameras in the system.

4. The surveillance system of claim 1 further comprises a threat detection algorithm embodied in software installed at the first camera for comparing object movements to the motion flow model.

5. The surveillance system of claim 1 wherein the motion flow model residing at the first camera defines directional paths for objects moving in the field of view of the first camera.

6. The surveillance system of claim 1 further comprises a remote monitoring station connected to the network and adapted to receive the alarm generated by the second camera.

28. The surveillance system of claim 1 wherein said tracking request message communicates to the second camera intermediate accumulative threat scores, calculated by the first camera to enable continued calculation of the threat score based on the second camera'"'"'s own assessment and based on threat scores of another camera.

29. The surveillance system of claim 1 wherein the second camera in turn passes back the accumulative threat scores to the first camera in the event an object moves back into the field of view of the first camera.

7. A method for detecting a threat in a distributed multi-camera surveillance system, comprising:

a plurality of cameras including;

training a statistical motion flow model of a first camera based on observed motion vectors within the first camera'"'"'s field of view in combination with feedback data compiled and communicated peer-to-peer to the camera by another camera in response to a tracking request message issued by the first camera, the tracking request message communicating time stamp and flow identifier information associated with a trained statistical motion flow, and the feedback data received by said another camera communicating matching time stamp and corresponding flow identifier information that is used by the first camera to dynamically learn a topological relationship between the first camera and said another camera;

wherein the motion flow model is a building algorithm embodied in software installed at each of the individual cameras of the plurality of cameras for constructing the motion flow model for their own independent fields of view and wherein the dynamic learning of the topological relationships between the cameras includes each individual camera dynamically updating its statistical model, in real-time, upon the receiving of the dynamically transmitted tracking requests transmitted by the other individual cameras, such that the individual cameras are able to dynamically and individually adapt to their individual orientations and positions;

the system further including;

monitoring movement of an object in a field of view of the first camera using software installed at the first camera that accesses the statistical motion flow model of the first camera;

detecting a threat at the first camera when movement of the object does not conform with a motion flow model residing at the first camera;

using said tracking request message for the object from the first camera to a second camera upon detecting the threat at the first camera;

monitoring movement of the object in a field of view of the second camera using software installed at the second camera;

detecting a threat at the second camera when the movement of the object does not conform with a motion flow model residing at the second camera; and

generating an alarm based in part on the threat detected at the first camera and the second camera.

8. The method of claim 7 wherein building the motion flow model further comprises:

defining tracks for objects moving in the field of view of the first camera using an object tracking algorithm;

determining entry areas and exit areas in the field of view of the first camera by applying a clustering algorithm to beginning points and end points of the tracks;

defining directional flows between the entry areas and the exit areas; and

assigning each track to a directional flow using a begin point and an end point associated with a given track.

9. The method of claim 7 further comprises learning motion flow of objects between the first camera and other cameras in the system, and storing camera associations in a data store at the first camera.

10. The method of claim 7 further comprises:

assigning a threat score for the object detected at the first camera;

sending the threat score in the tracking request from the first camera to the second camera; and

generating the alarm based in part on the threat score from the first camera.

11. The method of claim 7 further comprises assigning threat scores to the object detected at the second camera and generating the alarm based on the threat scores associated with the threat detected at the first and second cameras.

12. The method of claim 7 wherein generating an alarm further comprises sending an alarm message from the second camera to a remote monitoring location.

13. The method of claim 12 further comprises sending video data showing the object in a field of view of the first camera from the first camera over the network to the remote monitoring location and sending video data showing the object in the field of view of the second camera from the second camera over the network to the remote monitoring location.

14. The method of claim 7 further comprises:

sending a tracking request from the second camera to a third camera upon detecting the suspicious object at the second camera;

monitoring movement of the object in a field of view of the third camera using software installed at the third camera; and

generating an alarm when the movement of the object does not conform with a motion flow model residing at the third camera.

15. A method for learning motion flow of objects amongst two or more cameras in a distributed multi-camera surveillance system, comprising:

training a statistical motion flow model of a first camera based on observed motion vectors within the first camera'"'"'s field of view in combination with feedback data compiled and communicated peer-to-peer to the camera by another camera in response to a tracking request message issued by the first camera, the tracking request message communicating time stamp and flow identifier information associated with a trained statistical motion flow, and the feedback data received by said another camera communicating matching time stamp and corresponding flow identifier information that is used by the first camera to dynamically learn a topological relationship between the first camera and said another camera;

detecting an object moving in a field of view of a first camera using software installed at the first camera that accesses the statistical motion flow model of the first camera;

wherein the motion flow model is a building algorithm embodied in software installed at each of the individual cameras of the plurality of cameras for constructing the motion flow model for their own independent fields of view and wherein the dynamic learning of the topological relationships between the cameras includes each individual camera dynamically updating its statistical model, in real-time, upon the receiving of the dynamically transmitted tracking requests transmitted by the other individual cameras, such that the individual cameras are able to dynamically and individually adapt to their individual orientations and positions;

the system further including;

using said tracking request message from the first camera across a network to other cameras in the network, wherein the tracking request provides an identifier for the first camera, an identifier for the object and visual attributes associated with the object;

searching for the object in a field of view of a second camera in response to the tracking request received from the first camera; and

associating the second camera with the first camera when the object is detected in the field of view of the second camera.

16. The method of claim 15 further comprises searching for the object at the second camera during a window of time defined in the tracking request.

17. The method of claim 15 further comprises sending a message from the second camera to the first camera when the object is detected in the field of view of the second camera and associating the second camera with the first camera based on the message.

18. The method of claim 17 further comprises sending a subsequent tracking message from the first camera to the second camera based on the associate between the two cameras.

19. The method of claim 15 further comprises maintaining a confidence metric regarding the associate of the first camera with the second camera at the first camera and updating the confidence metric upon receipt of the message sent by the second camera to the first camera.

20. The method of claim 15 further comprises sending a message which includes a confidence measure that the detected object correlates to the object detected by the first camera from the second camera to the first camera.

21. The method of claim 15 further comprises receiving feedback messages at the first camera from multiple cameras, where each feedback message includes a confidence measure that the object detected by a given camera correlates to the object detected by the first camera, and associating the first camera with one of the multiple cameras based on the confidence measures.

22. The method of claim 15 further comprises reporting the associate between the first and second cameras to a remote monitoring location.

23. The method of claim 15 further comprises retrieving the associate between the first and second cameras from the first camera by a remote monitoring location.

24. The method of claim 15 further comprises learning where an object traversing a given flow in the field of the first camera will re-appear in the system.

25. The method of claim 24 further comprises maintaining a record in a list for each location the object re-appears and updating a probability the object appeared at each location based on feedback messages received from other cameras in the system.

26. The method of claim 15 further comprises learning where an object was prior to appearing in an entry region of its field of view.

27. The method of claim 26 further comprises maintaining a record in a list for each location the object was and updating a probability the object came from each location based on feedback messages received from other cameras in the system.