SYSTEM AND METHODS FOR IMPROVING ACCURACY AND ROBUSTNESS OF ABNORMAL BEHAVIOR DETECTION

• US 20100208063A1
• Filed: 07/02/2009
• Published: 08/19/2010
• Est. Priority Date: 02/19/2009
• Status: Abandoned Application

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Shi, Xiangjun, Ozdemir, Hasan Timucin, Lee, Kuo Chu, Yu, Juan
• Application Number:
  US12/496,681
• Publication Number:
  US 20100208063A1
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19641 Multiple cameras having overlapping views on a single scene
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance system improves accuracy and robustness of abnormal behavior detection of a monitored object traversing a space includes a metadata processing module, a model building module, and a behavior assessment module. The metadata processing module generates trajectory information for a monitor object and determines attributes of the monitored object. The model building module at least one of generates and updates normal motion models based on at least one of the trajectory information, the attributes, and an abnormal behavior score. The behavior assessment module generates the abnormal behavior score based on one of a plurality of methods. A first one of the plurality of methods defines wrong direction behavior. A second one of the plurality of methods defines wandering/loitering behavior. A third one of the plurality of methods defines speeding behavior.

Claims

1. A method for determining abnormal behavior of an object traversing a space, comprising:

receiving trajectory information for an object whose movement in the space is being monitored, where the trajectory information indicates a current position of the monitored object;

retrieving a trajectory model that corresponds to the current position of the monitored object, where the trajectory model defines possible directions that an object at the current position may travel and, for each possible direction, a likelihood that the object at the current position would travel in the corresponding possible direction;

computing a likelihood that the monitored object is traveling in a direction based on a weighted average of likelihoods for two or more of the possible directions given by the model, where the two or more possible directions are those nearest to the direction of the monitored object; and

identifying abnormal behavior of the monitored object based on the computed likelihood.

2. The method of claim 1, further comprising:

generating the trajectory information for the monitored object based on sensor data received from a plurality of sensing devices.

3. The method of claim 2, wherein the sensing devices are video cameras.

4. The method of claim 1, further comprising:

building the trajectory model based on past behavior of objects in the space.

5. The method of claim 4, further comprising:

updating the trajectory model based on at least one of the trajectory information for the monitored object and the identification of abnormal behavior.

6. The method of claim 1, wherein computing the likelihood that the monitored object is traveling in a direction further includes:

determining differences between a directional angle of the monitored object and directional angles of the possible directions.

7. The method of claim 6, wherein the two or more of the possible directions are correspond to smallest differences.

8. The method of claim 1, wherein computing the likelihood that the monitored object is traveling in a direction further includes:

generating a plurality of raw abnormality scores based on the likelihood for each of the possible directions; and

averaging the plurality of raw abnormality scores during a predetermined time period.

9. The method of claim 8, wherein identifying abnormal behavior of the monitored object based on the computed likelihood further includes:

actuating at least one of a graphical user interface, an audio/visual alarm, and a recording storage module based on the computed likelihood and a predetermined threshold.

45. The surveillance system of claim 32, wherein the abnormal behavior detection methods include the method of claim 1.

10. A method for determining abnormal behavior of an object traversing a space, comprising:

receiving trajectory information for an object whose movement in the space is being monitored, where the trajectory information indicates a current position of the monitored object and a distances that the monitored object has traveled from the current position during a previous time period;

retrieving a trajectory model that corresponds to the current position of the monitored object, where the trajectory model defines a threshold distance that an object at the current position would have traveled from the current position during the previous time period;

comparing the distances to the threshold distance; and

identifying abnormal behavior of the monitored object based on the comparison.

11. The method of claim 10, wherein the previous time period includes a plurality of samples based on a sampling rate.

12. The method of claim 11, wherein identifying abnormal behavior of the monitored object further includes:

comparing a distance of the monitored object from the current position at each of the plurality of samples to the threshold distance; and

incrementing a count when the distance is less than the threshold distance.

13. The method of claim 12, wherein identifying abnormal behavior of the monitored object further includes:

actuating at least one of a graphical user interface, an audio/visual alarm, and a recording storage module based on the count and a count threshold.

14. The method of claim 10, wherein identifying abnormal behavior of the monitored object further includes:

determining a dwell time period based on distances of the monitored object from the current position during the previous time period, wherein the dwell time period includes when the monitored object is less than the threshold distance from the current position.

15. The method of claim 14, wherein identifying abnormal behavior of the monitored object further includes:

generating a confidence factor based on a size of the monitored object in pixels and a predefined pixel size model.

16. The method of claim 15, wherein identifying abnormal behavior of the monitored object further includes:

actuating at least one of a graphical user interface, an audio/visual alarm, and a recording storage module based on the previous time period, the dwell time period, and the confidence factor.

17. The method of claim 10, further comprising:

generating the trajectory information for the monitored object based on sensor data received from a plurality of sensing devices.

18. The method of claim 17, wherein the sensing devices are video cameras.

19. The method of claim 10, further comprising:

building the trajectory model, wherein the threshold distance is based on and an average speed of the monitored object during the previous time period and an average direction of the monitored object during the previous time period

20. The method of claim 19, further comprising:

updating the trajectory model based on at least one of the trajectory information for the monitored object and the identification of abnormal behavior.

46. The surveillance system of claim 32, wherein the abnormal behavior detection methods include the method of claim 10.

21. A method for determining abnormal behavior of an object traversing a space, comprising:

receiving trajectory information for an object whose movement in the space is being monitored, where the trajectory information indicates a current position of the monitored object, a direction that the monitored object is traveling, and a velocity of the monitored object;

retrieving a trajectory model that corresponds to the current position of the monitored object, where the trajectory model defines possible directions that an object at the current position may travel and, for each possible direction, a velocity that the object at the current position would travel at;

computing a velocity threshold for the monitored object based on a weighted average of the velocities for two or more of the possible directions given by the model, where the two or more possible directions are those nearest to the direction of the monitored object; and

identifying abnormal behavior of the monitored object based on the velocity of the monitored object and the computed velocity threshold.

22. The method of claim 21, further comprising:

generating the trajectory information for the monitored object based on sensor data received from a plurality of sensing devices.

23. The method of claim 22, wherein the sensing devices are video cameras.

24. The method of claim 21, further comprising:

building the trajectory model based on past behavior of objects in the space.

25. The method of claim 24, further comprising:

updating the trajectory model based on at least one of the trajectory information for the monitored object and the identification of abnormal behavior.

26. The method of claim 21, wherein computing the velocity threshold for the monitored object further includes:

determining differences between a directional angle of the monitored object and directional angles of the possible directions.

27. The method of claim 26, wherein the two or more of the possible directions correspond to smallest differences.

28. The method of claim 27, wherein computing the velocity threshold for the monitored object further includes:

generating weight factors for each of the two or more possible directions based corresponding angle differences.

29. The method of claim 28, wherein computing the velocity threshold for the monitored object further includes:

determining an expected velocity of the monitored object based on the weight factors and the velocities corresponding to the two or more possible directions.

30. The method of claim 29, wherein computing the velocity threshold for the monitored object further includes:

generating a plurality of raw abnormality scores based on the expected velocity and the velocity of the monitored object; and

determining a median of the plurality of raw abnormality scores during a predetermined time period.

31. The method of claim 21, wherein identifying abnormal behavior of the monitored object based on the computed likelihood further includes:

actuating at least one of a graphical user interface, an audio/visual alarm, and a recording storage module based on the velocity of the monitored object and the computed velocity threshold.

47. The surveillance system of claim 32, wherein the abnormal behavior detection methods include the method of claim 21.

32. A surveillance system that improves accuracy and robustness of abnormal behavior detection of a monitored object traversing a space, comprising:

a metadata processing module that generates trajectory information corresponding to the monitored object and that determines attributes of the monitored object based on at least one of a plurality of normal motion models and a dynamic time window, wherein the attributes include an estimated velocity of the monitored object, whether the monitored object is an outlier, and a measurement error estimation;

a model building module that at least one of generates and updates the plurality of normal motion models based on at least one of the attributes of the monitored object and an abnormality score corresponding to the monitored object; and

a behavior assessment module that generates the abnormal behavior score corresponding to the monitored object based on one of a plurality of abnormal behavior detection methods.

33. The surveillance system of claim 32, further comprising:

a plurality of sensing devices that generate the trajectory information based on at least one of video data and metadata.

34. The surveillance system of claim 33, wherein the plurality of sensing devices are video cameras.

35. The surveillance system of claim 33, wherein the metadata processing module, the model building module, and the plurality of sensing devices are adapted via an open-interface to receive the output of an outlier handling function of the behavior assessment module, wherein the open-interface enables scalable abnormal behavior score implementation and extensible abnormal behavior detection.

36. The surveillance system of claim 33, wherein the plurality of sensing devices further include runtime abnormal behavior detection models that are installed on demand and start processing the metadata for generation of new types of abnormal behavior detection features.

37. The surveillance system of claim 32, wherein the attributes of the monitored object include a change in motion direction, a ratio of the estimated velocity over an estimated average velocity, and a randomness factor, wherein the estimated average velocity corresponds to one of the plurality of normal motion models, and wherein the randomness factor based on distance traveled and the total position estimation error within the dynamic time window.

38. The surveillance system of claim 37, wherein the monitored object is determined to be an outlier based on at least one of a change in pixel size of the monitored object during the dynamic time window, a change in a boundary box corresponding to the monitored object during the dynamic time window, and a change in position of the monitored object during the dynamic time window.

39. The surveillance system of claim 38, wherein accuracy of the abnormal behavior score increases based on whether the monitored object is an outlier, the estimated measurement error, and the randomness factor.

40. The surveillance system of claim 38, wherein accuracy of the abnormal behavior score increases based on the ratio of the estimated velocity over the estimated average velocity.

41. The surveillance system of claim 38, wherein the normal motion models are not updated with velocity information corresponding to the monitored object when an observation of the monitored object is determined to be an outlier

42. The surveillance system of claim 41, wherein the velocity information includes minimal displacement-based speeds in horizontal and vertical directions, a direction of the monitored object, and a change in motion direction corresponding to a previous position of the monitored object and a current position of the monitored object.

43. The surveillance system of claim 42, wherein the minimal displacement-based speeds in the horizontal and vertical directions are available when a distance to the current position of the monitored object is greater than a minimum distance, wherein the minimum distance is based on the pixel size of the monitored object and an external tracking algorithm.

44. The surveillance system of claim 32, further comprising:

an alarm generation module that actuates at least one of a graphical user interface, an audio/visual alarm, and a recording storage module based on the abnormal behavior score and a score threshold.