Apparatus and methods for the detection of abnormal motion in a video stream

• US 20060045185A1
• Filed: 08/31/2004
• Published: 03/02/2006
• Est. Priority Date: 08/31/2004
• Status: Active Grant

• Current Assignee: Qognify Ltd. (PSBU Midco SARL), Ramot at Tel Aviv University Limited (Tel Aviv University)
• Original Assignee: Nice Systems Limited (Nice Ltd), Ramot at Tel Aviv University Limited (Tel Aviv University)
• Inventors: Harari, Daniel, Ivanchenko, Yan, Rochel, Shay, Riklin-Raviv, Tamar, Kiryati, Nahum, Dvir, Igal
• Granted Patent: US 8,724,891 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An apparatus and method for detection of abnormal motion in video stream, comprising a training phase for defining normal motion and a detection phase for detecting abnormal motions in the video stream. Motion is detected according to motion vectors and motion features extracted from video frames.

Claims

1. An apparatus for detection of abnormal motion in video stream, the video stream comprises at least one video frame, said at least one video frame comprises:

at least one motion vector, each at least one motion vector representing the approximate common movement direction of a sub-part of said at least one video frame;

the apparatus comprises;

an extraction module for extracting at least one motion vector from the at least one video frame, each at least one motion vector representing the approximate common movement direction of a sub-part of said at least one video frame;

a comparison module for

receiving at least one statistical model representing motion in the scene captured by the at least one video frame; and

detecting abnormal motion in the at least one video frame based on the relation between the at least one motion vector of the at least one video frame and an at least one statistical model.

2. The apparatus of claim 1 wherein the at least one statistical model represents normal motion at the site covered by the at least one video frame.

3. The apparatus of claim 2 wherein the comparison module detects abnormal motion based on the mismatch between the at least one motion vector of the at least one video frame and the at least one statistical model.

4. The apparatus of claim 1 wherein the at least one statistical model represents abnormal motion at the site covered by the at least one motion vector of the at least one video frame.

5. The apparatus of claim 4 wherein the comparison module detects abnormal motion based on the correspondence between the at least one motion vector of the at least one video frame and the at least one statistical model.

6. The apparatus of claim 1 wherein said at least one statistical model is based on at least one motion feature of the at least one video frame captured in association with the site covered by the at least one video frame.

7. The apparatus of claim 6 wherein the comparison module extracts said at least one motion feature from the at least one motion vector of the at least one video frame.

8. The apparatus of claim 6, wherein the at least one motion feature comprise any one of the following:

sum of absolute value of motion over the sub-parts within the at least one video frame;

index of region within the at least one video frame where the largest part of the motion takes place;

the largest part of the overall motion within the at least one video frame, occurring in a region;

index of angle range in which the absolute sum of the motion is largest;

the part of the total motion occurring in said angle range out of the total motion.

9. The apparatus of claim 6 where at least one motion feature includes time dependent information extracted from at least two video frames.

10. The apparatus of claim 1 where the at least one motion vector is quantized to limit number of possible values of said motion vector'"'"'s elements.

11. The apparatus of claim 1 where the at least one motion vector is filtered to reduce errors.

12. The apparatus of claim 1 or 6, further comprising a training module for creating the at least one statistical model, based on at least one motion vector or at least one motion feature representing motion.

13. The apparatus of claim 12 wherein the training module performs probability distribution estimation.

14. The apparatus of claim 12 wherein the training module generates at least one one-dimensional histogram representing the distribution of values of one of the at least one motion feature.

15. The apparatus of claim 12 wherein the training module generates a multi-dimensional histogram, each dimension of the multi-dimensional histogram representing the distribution of values of one of the at least one motion feature.

16. The apparatus of claim 12 wherein the training module generates a Gaussian type statistical model representing the distribution of an at least one motion feature.

17. The apparatus of claim 12 wherein the training module uses a k-means method to generate a statistical model representing the distribution of an at least one motion feature.

18. The apparatus of claim 1 further comprising a mechanism for generating an alert when abnormal motion is detected.

19. The apparatus of claim 18 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.

20. The apparatus of claim 1 wherein the statistical model is predetermined.

21. The apparatus of claim 1 further comprising a system maintenance and setup module for initializing the system parameters and maintaining the system.

22. The apparatus of claim 1 where the video stream comprises at least one compressed video frame, said at least one compressed video frame comprises at least one motion vector, each at least one motion vector representing the approximate common movement direction of a sub-part of said at least one compressed video frame.

23. An apparatus comprising at least two multiple instances of the apparatus of claim 1 and a control unit where the control unit grades the severity of abnormal motions detected by said instances and generates an alert for the at least one most abnormal motion detected by said instances.

24. The apparatus of claim 23 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.

25. A method for detecting abnormal motion in video stream, the video stream comprises at least one video frame;

the method comprises the steps of

extracting an at least one motion vector, each at least one motion vector representing the approximate common movement of a sub-part of said at least one video frame;

comparing the at least one motion vector of the at least one video frame to an at least one statistical model, each said at least one statistical model representing motion in the site depicted by the video stream.

26. The method of claim 25 wherein the at least one statistical model represents normal motion in the site depicted by the video stream.

27. The method of claim 26 further comprising the step of detecting the mismatch between said at least one statistical model and said at least one motion vector of said at least one video frame.

28. The method of claim 25 where at least one statistical model represents abnormal motion in the site depicted by the video stream.

29. The method of claim 28 further comprising the step of detecting the correspondence between said at least one statistical model and said at least one motion vector of said at least one video frame.

30. The method of claim 25 wherein each at least one statistical model is based on at least one motion feature of the at least one video frame in the site covered by the at least one video frame.

31. The method of claim 30 further comprising the steps of:

extracting said at least one motion feature of said at least one video frame; and

comparing said at least one motion feature of said at least one statistical model to said at least one motion feature of said at least one video frame.

32. The method of claim 30 where the at least one motion feature comprise any of the following:

sum of absolute value of motion over all macro blocks;

index of region within the at least one video frame where the largest part of the motion takes place;

the largest part of the overall motion within the at least one video frame, occurring in a specific region;

index of angle range in which the absolute sum of the motion is largest;

the part of the total motion occurring in this angle range out of the total motion.

33. The method of claim 30 where at least one motion feature includes time dependent information extracted from at least two video frames.

34. The method of claim 25 further comprising the step of quantizing at least one motion vector to limit number of possible values of said motion vector'"'"'s elements.

35. The method of claim 25 further comprising the step of filtering at least one motion vector to reduce errors.

36. The method of claim 25 further comprising the step of training for creating the at least one statistical model, based on a video stream representing motion at the scene covered by the at least one video frame, the method comprising the steps of

calculating the motion features for each at least one video frame;

and

building the statistical model based on the motion features of the at least one video frame.

37. The method of claim 36 wherein the training module performs probability distribution estimation.

38. The method of claim 36 wherein the training module generates at least one one-dimensional histogram representing the distribution of values of the at least one motion feature.

39. The method of claim 36 wherein the training module generates a multi-dimensional histogram, each dimension of the multi-dimensional histogram representing the distribution of values of one of the at least one motion feature.

40. The method of claim 36 wherein the training module generates a Gaussian type statistical model representing the distribution of an at least one motion feature.

41. The method of claim 36 wherein the training module generates a k-means statistical model representing the distribution of an at least one motion feature.

42. The method of claim 25 further comprising the step of generating an alert when abnormal motion is detected.

43. The method of claim 42 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.

44. The method of claim 25 wherein the statistical model is predetermined.

45. The method of claim 25 where the video stream comprises at least one compressed video frame, each at least one compressed video frame comprises at least one motion vector, each at least one motion vector representing the approximate common movement of a sub-part of said at least one compressed video frame.

46. A method of grading the severity of abnormal motion detected in at least two video streams by the method of claim 25.

47. The method of claim 46 further comprising a step of generating an alert for the said at least one most severe abnormal motion.

48. The method of claim 47 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.