Method and apparatus for video frame sequence-based object tracking

• US 20040240542A1
• Filed: 12/10/2003
• Published: 12/02/2004
• Est. Priority Date: 02/06/2002
• Status: Active Grant

• Current Assignee: Qognify Ltd. (PSBU Midco SARL)
• Original Assignee: Nice Systems Limited (Nice Ltd)
• Inventors: Dvir, Igal, Yeredor, Arie, Koren-Blumstein, Guy, Lachover, Boaz
• Granted Patent: US 7,436,887 B2
• Time in Patent Office: Days
• US Class Current: 375/240.01
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19615 wherein said pattern is defined by the user
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system

Abstract

A surveillance system and method for the automatic detection of potential alarm situation via a recorded surveillance content analysis and for the management of the detected unattended object situation. The proposed system and method is operative in capturing surveillance content, analyzing the captured content and providing in real time a set of alarm messages to a set of diverse devices. The system provides event based debriefing according to captured objects captured by one or more cameras covering different scenes. The invention is implemented in the context of unattended objects (such as luggage, vehicles or persons), parking or driving in restricted zones, controlling access of persons into restricted zones, preventing loss of objects such as luggage or persons and counting of persons.

Claims

1. A method for analyzing video data, comprising:

receiving a video frame;

comparing said video frame to at least one background reference frame to locate at least one difference;

locating a plurality of objects to form a plurality of marked objects; and

determining a behavior pattern for at least one object according to said at least one difference, wherein said behavior pattern is defined according at least one scene characteristic.

2. The method of claim 1, further comprising producing at least one updated background reference frame.

3. The method of claim 1, wherein determining said at least one difference is performed by creating a difference frame between said video frame and said at least one background reference frame.

4. The method of claim 1, further comprising finding at least one new object when determining said at least one difference.

5. The method of claim 1, further comprising issuing an alarm according to said behavior pattern.

6. The method as claimed in claim 1 wherein a pre-defined pattern of suspicious behavior comprises an object presenting unpredictable behavior.

7. A system for analyzing video data comprising a plurality of video frames, the system comprising:

a video frame preprocessing layer for determining a difference between a plurality of video frames;

an object clustering layer for detecting a plurality of objects according to said difference; and

an application layer for characterizing said plurality of objects according to at least one scene characteristic.

8. The system of claim 7, wherein said difference is determined between a video frame and at least one reference frame, the system further comprising a background refreshing layer for preparing at least one updated reference frame according to the said difference.

9. The system of claim 8, wherein said at least one scene characteristic defines a behavior pattern for at least one object, such that if said at least one object exhibits said behavior pattern, said at least one scene characteristic is detected.

10. The system of claim 9, wherein if said at least one scene characteristic is detected, an alarm is generated.

11. The system of claim 9, wherein said at least one scene characteristic further comprises at least one parameter for determining if said at least one object exhibits said behavior pattern.

12. A system for detecting a vehicle remaining in a restricted zone for at least a minimum period of time, comprising:

a video content analysis module for analyzing video data of the restricted zone, said video content analysis module further comprising an object tracking component; and

an application layer for receiving data from said video content analysis module and for detecting a vehicle remaining in the restricted zone for at least the minimum period of time, and said application layer generating an alarm upon detection.

13. A system for detecting unattended luggage, bag or any unattended object in an area, comprising:

a video content analysis module for analyzing video data of the area, said video content analysis module further comprising an object tracking component; and

an application layer for receiving data from said video content analysis module and for detecting an unattended object, wherein said unattended object has not been attended in the area for more then a predefined period of time.

14. A surveillance system for the detection of an alarm situation, the system comprising the elements of:

a video analysis unit for analyzing video data representing images of a monitored area, the video analysis unit comprising an object tracking module to track the movements and the location of a video object;

a detection, surveillance and alarm application for receiving video data analysis results from the video analysis unit, for identifying an alarm situation and to generate an alarm signal;

an events database to hold video objects, video object parameters and events identified by the application.

15. The system of claim 14 further comprises the elements of:

an application driver to control the detection, surveillance and alarm application;

a database handler to access, to update and to read the events database;

a user interface component to communicate with a user of the system;

an application setup and control component to define the control parameters of the application;

an application setup parameters table to store the control parameters of the application.

16. The system as claimed in claim 14 further comprises the elements of:

a video data recording and compression unit to record and compress video data representing images of a monitored area;

a video archive file to hold the recorded and compressed video data representing images of the a monitored area;

an alarm distribution unit to distribute the alarm signal representing an alarm situation.

17. The system as claimed in claim 14 further comprises the elements of:

at least one video camera to obtain the images of a monitored area;

at least one a video capture component to capture video data representative of the images of the monitored area;

at least one video transfer component to transfer the captured video data to the video analysis unit and the recording compressing and archiving unit;

at least one computing and storage device.

18. The system as claimed in claim 14 wherein the object tracking module comprises the elements of:

a video frame preprocessing layer for determining the difference between at least two video frames;

an objects clustering layer for detecting at least one objects in accordance with the determined difference;

a scene characterization layer for characterizing the a least one object according to least one characteristic of a scene;

a background refreshing layer for preparing at least one updated reference according to the determined difference.

19. The system as claimed in claim 14 wherein the detection surveillance and alarm application is operative in the detection of at least one unattended object in the monitored area.

20. The system as claimed in claim 17 wherein any of the at least one video camera;

the at least one video capturing component;

the at least one video transfer component and the at least one computing and storage device can be separated and can be located in different locations.

21. The system as claimed in claim 17 wherein the interface between the at least one video camera;

the at least one video capturing component;

the at least one video transfer component and the at least one computing and storage device is a local or wide area network or a packed-based or cellular or radio frequency or micro wave or satellite network.

22. The system as claimed in claim 19 wherein the at least one unattended object is a luggage left in an airport terminal for a pre-determined period.

23. The system as claimed in claim 22 the at least one unattended object is a vehicle parking in a restricted zone for a pre-defined period.

24. The system as claimed in claim 14 wherein the detection surveillance and alarm application is operative in the detection of an unpredicted object movement.

25. The system as claimed in claim 14 wherein the analysis is also performed on audio data or thermal imaging data or radio frequency data associated with the video data or the video object in synchronization with the video data.

26. The system as claimed in claim 17 wherein the video capture component captures audio or thermal information or radio frequency information in synchronization with the video data.

27. A surveillance method for the detection of an alarm situation, the surveillance to be performed on at least one monitored scene having at least one camera, the method comprising the steps of:

obtaining video data from the at least one camera representing images of a the at least one monitored scene;

analyzing the obtained video data representing images of the at least one object within the at least one monitored scene, the analyzing step comprising of identifying the at least one object within the video data; and

inserting the identified at least one object and the at least one event into an event database.

28. The method of claim 27 further comprising the steps of:

retrieving at least one of the object associated with at least one event;

according to user instruction displaying the video event associated with the retrieved at least one object.

29. The method of claim 27 further comprising the steps of:

retrieving at least two events;

associating according to parameters of the at least one object, the at least one object with the at least two events.

30. The method of claim 27 further comprising the steps of:

debriefing the at least one object associated with the at least one event to identify the pattern of behavior or movement of the at least one object within the at least one scene within a predefined period of time.

31. The method of claim 27 further comprising the steps of:

pre-defining patterns of suspicious behavior; and

pre-defining control parameters.

32. The method of claim 27 further comprising the steps of:

recognizing an alarm situation according to the pre-defined patterns of suspicious behavior; and

generating an alarm signal associated with the recognized alarm situation.

34. The method as claimed in claim 27 wherein a pre-defined pattern of suspicious behavior comprises:

an object entering a monitored scene;

the object separating into a first distinct object and a second distinct object in the monitored scene;

the first distinct object remaining in the monitored scene without movement for a pre-defined period; and

the second distinct object leaving the monitored scene.

35. The method of claimed in claim 34 wherein the pre-defined pattern of suspicious behavior comprises:

an object entering the monitored scene;

the object ceasing its movement;

the size of the object is recognized as being above a pre-defined parameter value; and

the object remaining immobile for a period recognized as being above a pre-defined parameter value.

36. The method of claim 35 further comprising identifying information associated with the object for the purpose of identifying the at least one object.

33. The method as claimed in 27 further comprises the steps of:

implementing patterns of suspicious behavior introducing pre-defined control parameters;

recording, compressing and archiving the obtained video data;

distributing the alarm signal representing an alarm situation across a pre-defined range of user devices.