System and method for video content analysis-based detection, surveillance and alarm management

• US 7,683,929 B2
• Filed: 12/26/2002
• Issued: 03/23/2010
• Est. Priority Date: 02/06/2002
• Status: Active Grant

• Current Assignee: Qognify Ltd. (PSBU Midco SARL)
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Dvir, Igal, Koren-Blumstein, Guy, Elazar, Avishai, Back, Ayelet
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US10/478,577
• Publication Number:
  US 20040161133A1
• Time in Patent Office: 2,644 Days
• Field of Search: 348/143, 348/148, 348/169, 348/154, 382/181, 382/103, 715/251
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A surveillance system and method for the automatic detection of potential alarm situation via a recorded surveillance content analysis and for the management of the detected unattended object situation. The system and method are operative in capturing surveillance content, analyzing the captured content and providing in real time a set of alarm messages to a set of diverse devices. The system provides event based debriefing according to captured objects captured by one or more cameras covering different scenes. The invention is implemented in the context of unattended objects (such as luggage, vehicles or persons), parking or driving in restricted zones, controlling access of persons into restricted zones, preventing loss of objects such as luggage or persons and counting of persons.

Claims

1. A surveillance system for the detection of an alarm situation, the system comprising the elements of:

a video analysis unit for analyzing at least two video data streams recorded by at least two capturing devices, each of the two streams representing images of a monitored area, the video analysis unit comprising an object tracking module for tracking the movements and the location of at least two video objects seen in the at least two video data streams;

a detection, surveillance and alarm application for receiving video data analysis results from the video analysis unit, for automatically identifying a first event associated with a first video object in a first video data stream, identifying a second event associated with a second video object a in a second video data stream, and linking the first event and the second event through association of the first object and the second object, and for generating an alarm signal; and

an event database for storing the first video object, the second video object, video object parameters, the first event and the second event,

wherein tracking the movements and the location of each of the at least two video objects is done by generating difference frames representing the differences between at least two reference frames and another frame from one of the at least two video data streams.

2. The system of claim 1 further comprises the elements of:

an application driver to control the detection, surveillance and alarm application;

a database handler to access, to update and to read the event database;

a user interface component to communicate with a user of the system;

an application setup and control component to define the control parameters of the application; and

an application setup parameters table to store the control parameters of the application.

3. The system as claimed in claim 1 further comprises the elements of:

a video data recording and compression unit to record and compress video data representing images of a monitored area;

a video archive file to hold the recorded and compressed video data representing images of the a monitored area; and

an alarm distribution unit to distribute the alarm signal representing an alarm situation to at least one alarm and messaging device.

4. The system as claimed in claim 1 further comprises the elements of:

at least two video cameras to obtain the images of a monitored area;

at least one video capture component to capture video data representative of the images of the monitored area;

at least one video transfer component to transfer the captured video data to the video analysis unit and the recording compressing and archiving unit; and

at least one computing and storage device.

5. The system as claimed in claim 4 wherein any of the at least two video cameras;

the at least one video capturing component;

the at least one video transfer component and the at least one computing and storage device can be separated and can be located in different locations.

6. The system as claimed in claim 4 wherein the video capture component captures audio or thermal information or radio frequency information in synchronization with the video data.

7. The system as claimed in claim 4 wherein the interface between the at least two video cameras;

the at least one video capturing component;

the at least one video transfer component and the at least one computing and storage device is a local or wide area network or a packed-based or cellular or radio frequency or micro wave or satellite network.

8. The system as claimed in claim 1 wherein the object tracking module comprises the elements of:

a video frame preprocessing layer for determining the difference between at least two video frames;

an objects clustering layer for detecting at least one object in accordance with the determined difference;

a scene characterization layer for characterizing the a least one object according to least one characteristic of a scene; and

a background refreshing layer for preparing at least one updated reference according to the determined difference.

9. The system as claimed in claim 1 wherein the detection surveillance and alarm application is operative in the detection of at least one unattended object in the monitored area.

10. The system as claimed in claim 9 wherein the at least one unattended object is a luggage left in an airport terminal for a pre-determined period.

11. The system as claimed in claim 10 the at least one unattended object is a vehicle parking in a restricted zone for a pre-defined period.

12. The system as claimed in claim 1 wherein the detection surveillance and alarm application is operative in the detection of an unpredicted object movement.

13. The system as claimed in claim 1 wherein the analysis is also performed on audio data or thermal imaging data or radio frequency data associated with the video data or the first video object or the second video object in synchronization with the video data.

14. The system of claim 1 wherein the detection surveillance and alarm application is operative in the detection of a video object moving in a direction that does not match a predefined flow direction.

15. The system of claim 1, wherein the first event or the second event is identified using a size of the video object, and at least one parameter not related to the object.

16. The system of claim 15 wherein the at least one parameter is selected from the group consisting of:

search area and time out.

17. The system of claim 1 wherein linking the first event and the second event is done in accordance with the first object'"'"'s and the second object'"'"'s oval characteristics and location.

18. The system of claim 1 wherein the at least two reference frames are captured at least two capture devices.

19. A surveillance method for the detection of an alarm situation, the surveillance to be performed on at least one monitored scene having at least two cameras, the method comprising the steps of:

obtaining video data streams from the at least two cameras representing images of the at least one monitored scene;

analyzing the obtained video data streams representing images of at least one object within the at least one monitored scene, the analyzing step comprising the steps of identifying a first video object associated with a first event in a first video data stream and identifying a second video object associated with a second event in a second video data stream;

linking the first event and the second event through association of the first object and the second object; and

inserting the first video object, the second video object, the first event and the second event into an event database,

wherein tracking the movements and the location of each of the at least two video objects is done by generating difference frames representing the differences between at least two reference frames and another frame from one of the at least two video data streams.

20. The method of claim 19 further comprising the steps of:

retrieving the first video object or the second video object associated with the first event or the second event; and

according to user instruction displaying the first event or the second event associated with the first video object or the second video object.

21. The method of claim 19 wherein linking the first event and the second event comprises the steps of:

retrieving the first event and the second event; and

linking according to parameters of the first video object and the second video object, the first event with the second event.

22. The method of claim 19 further comprising the steps of:

debriefing the first object associated with the first event to identify the pattern of behavior or movement of the first object within the at least one scene within a predefined period of time.

23. The method of claim 19 further comprising the steps of:

pre-defining patterns of suspicious behavior; and

pre-defining control parameters.

24. The method of claim 19 further comprising the steps of:

recognizing an alarm situation according to the pre-defined patterns of suspicious behavior; and

generating an alarm signal associated with the recognized alarm situation.

25. The method of claim 19 further comprises the steps of:

implementing patterns of suspicious behavior;

introducing pre-defined control parameters;

recording, compressing and archiving the obtained video data; and

distributing the alarm signal representing an alarm situation to at least one alarm and messaging device, across a pre-defined range of user devices.

26. The method as claimed in claim 19 wherein a pre-defined pattern of suspicious behavior comprises:

an object entering a monitored scene;

the object separating into a first distinct object and a second distinct object in the monitored scene;

the first distinct object remaining in the monitored scene without movement for a pre-defined period; and

the second distinct object leaving the monitored scene.

27. The method of claimed in claim 26 wherein the pre-defined pattern of suspicious behavior comprises:

an object entering the monitored scene;

the object ceasing its movement;

the size of the object is recognized as being above a pre-defined parameter value; and

the object remaining immobile for a period recognized as being above a pre-defined parameter value.

28. The method of claim 27 further comprising identifying information associated with the object for the purpose of identifying the at least one object.

29. The method of claim 27 further comprising receiving data from the video analysis unit and detecting within the video data a vehicle remaining in a restricted zone for at least a predefined period of time.

30. The method of claim 29 wherein detecting the vehicle remaining in the restricted zone is performed using a size of the object, and at least one parameter and at least one parameter not related to the object.

31. The method of claim 30 wherein the at least one parameter is selected from the group consisting of:

search area and time out.

32. The method of claim 26 further comprising receiving data from the video analysis unit and detecting within the video data an unattended object within an area, wherein said unattended object has been unattended in the area for at least a predefined period of time.

33. The method of claim 32 wherein detecting the unattended object is performed using a size of the object, and at least one parameter not related to the object.

34. The method of claim 33 wherein the at least one parameter is selected from the group consisting of:

search are and time out.

35. The method of claim 19 wherein the analysis comprises receiving data from the video analysis unit and detecting within the video data an object within an area, wherein said object moves in a direction that does not match a predefined flow direction.

36. The method of claim 19, wherein the first event or the second event is identified using a size of the video object, and at least one parameter not related to the object.

37. The method of claim 36, wherein the at least one parameter is selected from the group consisting of:

search area and time out.

38. The method of claim 19 wherein linking the first event and the second event is done in accordance with the first object'"'"'s and the second object'"'"'s oval characteristics and location.

39. The method of claim 19 wherein the at least two reference frames are captured at least two capture devices.