Apparatus and methods for the detection of abnormal motion in a video stream

• US 8,724,891 B2
• Filed: 08/31/2004
• Issued: 05/13/2014
• Est. Priority Date: 08/31/2004
• Status: Active Grant

• Current Assignee: Qognify Ltd. (PSBU Midco SARL), Ramot at Tel Aviv University Limited (Tel Aviv University)
• Original Assignee: Nice Systems Limited (Nice Ltd), Ramot at Tel Aviv University Limited (Tel Aviv University)
• Inventors: Kiryati, Nahum, Riklin-Raviv, Tamar, Ivanchenko, Yan, Rochel, Shay, Dvir, Igal, Harari, Daniel
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Werner, David N
• Application Number:
  US10/930,254
• Publication Number:
  US 20060045185A1
• Time in Patent Office: 3,542 Days
• Field of Search: 348152-155, 382/155, 375/240.16
• US Class Current: 382/160
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: G08B 13/1968 Interfaces for setting up or customising the system
• CPC Class Codes: G08B 13/19691 Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An apparatus and method for detection of abnormal motion in video stream, having a training phase for defining normal motion and a detection phase for detecting abnormal motions in the video stream is provided. Motion is detected according to motion vectors and motion features extracted from video frames.

Claims

1. An apparatus for detection of abnormal activity in a video stream, the video stream having video frames, the apparatus comprising:

a system maintenance and setup module;

a module that extracts motion vectors, from each of the video frames, that represent an approximate common movement direction of a predetermined sub-part of each of said frames;

a system training module that creates at least one statistical model representing usual activity based on said motion vectors or on at least one motion feature generated from said motion vectors; and

a motion detection module that detects abnormal activity by determining according to the at least one statistical model, that a probability of at least one motion vector of said motion vectors associated with said predetermined sub-part or said at least one motion feature extracted from the at least one motion vector associated with said predetermined sub-part, is below a predetermined threshold when compared to the said at least one statistical model,

wherein said predetermined sub-part is a macro-block; and

wherein the at least one statistical model is represented as a one-dimensional histogram representing the distribution of values of one of the at least one motion feature.

2. The apparatus of claim 1, wherein the at least one motion feature comprises any one of the following:

sum of absolute value of motion over the sub-parts within the at least one video frame;

index of region within the at least one video frame where the largest part of the motion takes place;

the largest part of the overall motion within the at least one video frame, occurring in a region;

index of an angle range in which the absolute sum of the motion is largest;

or

the part of the total motion occurring in said angle range out of the total motion.

3. The apparatus of claim 1 wherein the at least one motion vector is filtered, by applying a filter selected from the group consisting of:

(a) a spike reducing filter, (b) a smoothing filter, and (c) an outlier removal filter, to reduce errors.

4. The apparatus of claim 1 further comprising an abnormal motion alert device for generating an alert when an abnormal motion is detected.

5. The apparatus of claim 4 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.

6. An apparatus comprising at least two instances of the apparatus of claim 1;

and a grading unit for grading the severity of abnormal motions detected by the at least two instances of the apparatus of claim 1 and for generating an alert for the at least one most abnormal motion detected by the at least two instances.

7. The apparatus of claim 6 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.

8. The apparatus of claim 1 further comprising:

at least one statistical model representing abnormal motion, and wherein;

the motion detection module compares at least one motion vector associated with the motion, or at least one motion feature extracted from the at least one motion vector associated with the motion, to the at least one statistical model representing abnormal motion.

9. An apparatus for detection of abnormal activity in a video stream, the video stream having video frames, the apparatus comprising:

a system maintenance and setup module;

a module that extracts motion vectors, from each of the video frames, that represent an approximate common movement direction of a predetermined sub-part of each of said frames;

a system training module that creates at least one statistical model representing usual activity based on said motion vectors or on at least one motion feature generated from said motion vectors; and

a motion detection module that detects abnormal activity by determining according to the at least one statistical model, that a probability of at least one motion vector of said motion vectors associated with said predetermined sub-part or said at least one motion feature extracted from the at least one motion vector associated with said predetermined sub-part, is below a predetermined threshold when compared to the said at least one statistical model,

wherein said predetermined sub-part is a macro-block; and

wherein the at least one statistical model is represented as a multi-dimensional histogram, wherein each dimension of the multi-dimensional histogram represents a distribution of values of one of the at least one motion feature.

10. An apparatus for detection of abnormal activity in a video stream, the video stream having video frames, the apparatus comprising:

a system maintenance and setup module;

a module that extracts motion vectors, from each of the video frames, that represent an approximate common movement direction of a predetermined sub-part of each of said frames;

a system training module that creates at least one statistical model representing usual activity based on said motion vectors or on at least one motion feature generated from said motion vectors; and

a motion detection module that detects abnormal activity by determining according to the at least one statistical model, that a probability of at least one motion vector of said motion vectors associated with said predetermined sub-part or said at least one motion feature extracted from the at least one motion vector associated with said predetermined sub-part, is below a predetermined threshold when compared to the said at least one statistical model,

wherein said predetermined sub-part is a macro-block; and

wherein the statistical model representing the distribution of an at least one motion feature is created using a k-means method.

11. A method for detection of abnormal activity in a video stream, the video stream having video frames, the method comprising:

capturing frames with a video camera;

extracting motion vectors, from each of the frames, that represent an approximate common movement direction of a predetermined sub-part of said frames;

creating at least one statistical model, representing usual activity, based on said motion vectors, or on an at least one motion feature generated from said motion vectors;

detecting abnormal activity by determining according to the at least one statistical model that a probability of at least one motion vector associated with the said predetermined sub-part or at least one motion feature extracted from the at least one motion vector associated with said predetermined sub-part, is below a predetermined threshold; and

relaying a warning indication for the abnormal activity to a warning device,

wherein the sub-part is a macro-block; and

wherein the at least one statistical model is a one-dimensional histogram representing the distribution of values of the at least one motion feature.

12. The method of claim 11 wherein the at least one motion feature comprises any of the following:

sum of absolute value of motion over all macro blocks;

index of region within the at least one video frame where the largest part of the motion takes place;

the largest part of the overall motion within the at least one video frame, occurring in a specific region;

index of an angle range in which the absolute sum of the motion is largest;

the part of the total motion occurring in said angle range out of the total motion.

13. The method of claim 11 further comprising:

applying a filter to at least one motion vector to reduce errors,

wherein said filter is selected from the group consisting of;

(a) a spike reducing filter, (b) a smoothing filter, and (c) an outlier removal filter.

14. The method of claim 11 further comprising an abnormal activity alert generation step of generating an alert when abnormal activity is detected.

15. The method of claim 14 wherein the alert is any one of the following:

an audio indication, a visual indication, a message to be sent to a predetermined person or system, an instruction sent to a system for performing a step associated with said alarm.

16. The method of claim 11 further comprising a step of grading the severity of abnormal activities detected in at least two video streams.

17. The method of claim 16 further comprising a step of generating an alert for an abnormal activity that is a most severe abnormal activity among the abnormal activities detected in the at least two video streams.

18. The method of claim 11 wherein the statistical model represents abnormal motion and wherein said step of determining if a motion is normal or abnormal compares at least one motion feature, extracted from the motion vectors associated with the motion, to the at least one statistical model representing abnormal motion.

19. A method for detection of abnormal activity in a video stream, the video stream having video frames, the method comprising:

capturing frames with a video camera;

extracting motion vectors, from each of the frames, that represent an approximate common movement direction of a predetermined sub-part of said frames;

creating at least one statistical model, representing usual activity, based on said motion vectors, or on an at least one motion feature generated from said motion vectors;

detecting abnormal activity by determining according to the at least one statistical model that a probability of at least one motion vector associated with the said predetermined sub-part or at least one motion feature extracted from the at least one motion vector associated with said predetermined sub-part, is below a predetermined threshold; and

relaying a warning indication for the abnormal activity to a warning device,

wherein the sub-part is a macro-block; and

wherein the at least one statistical model is a multi-dimensional histogram, wherein each dimension of the multi-dimensional histogram represents a distribution of values of one of the at least one motion feature.

20. A method for detection of abnormal activity in a video stream, the video stream having video frames, the method comprising:

capturing frames with a video camera;

extracting motion vectors, from each of the frames, that represent an approximate common movement direction of a predetermined sub-part of said frames;

creating at least one statistical model, representing usual activity, based on said motion vectors, or on an at least one motion feature generated from said motion vectors;

detecting abnormal activity by determining according to the at least one statistical model that a probability of at least one motion vector associated with the said predetermined sub-part or at least one motion feature extracted from the at least one motion vector associated with said predetermined sub-part, is below a predetermined threshold; and

relaying a warning indication for the abnormal activity to a warning device,

wherein the sub-part is a macro-block; and

wherein the statistical model representing the distribution of an at least one motion feature is created using a k-means method.