Unsupervised learning of temporal anomalies for a video surveillance system
First Claim
1. A computer-implemented method for analyzing a sequence of video frames depicting a scene captured by a video camera, the method comprising:
- receiving a set of kinematic data derived by a computer vision engine observing a foreground object in one of the frames of video;
receiving temporal data specifying when the foreground object was observed in one of the frames of video;
passing the set of kinematic data and the temporal data to an adaptive resonance theory (ART) network, wherein the ART network models observed behavior of a plurality of foreground objects observed in the scene, relative to the kinematic data and the temporal data;
evaluating one or more clusters of the ART network to determine whether the set of kinematic data and temporal data passed to the ART network are indicative of an occurrence of a temporally anomaly; and
upon determining a temporal anomaly has occurred, publishing an alert message.
6 Assignments
0 Petitions
Accused Products
Abstract
Techniques are described for analyzing a stream of video frames to identify temporal anomalies. A video surveillance system configured to identify when agents depicted in the video stream engage in anomalous behavior, relative to the time-of-day (TOD) or day-of-week (DOW) at which the behavior occurs. A machine-learning engine may establish the normalcy of a scene by observing the scene over a specified period of time. Once the observations of the scene have matured, the actions of agents in the scene may be evaluated and classified as normal or abnormal temporal behavior, relative to the past observations.
63 Citations
25 Claims
-
1. A computer-implemented method for analyzing a sequence of video frames depicting a scene captured by a video camera, the method comprising:
-
receiving a set of kinematic data derived by a computer vision engine observing a foreground object in one of the frames of video; receiving temporal data specifying when the foreground object was observed in one of the frames of video; passing the set of kinematic data and the temporal data to an adaptive resonance theory (ART) network, wherein the ART network models observed behavior of a plurality of foreground objects observed in the scene, relative to the kinematic data and the temporal data; evaluating one or more clusters of the ART network to determine whether the set of kinematic data and temporal data passed to the ART network are indicative of an occurrence of a temporally anomaly; and upon determining a temporal anomaly has occurred, publishing an alert message. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for analyzing a sequence of video frames depicting a scene captured by a video camera, the operation comprising:
-
receiving a set of kinematic data derived by a computer vision engine observing a foreground object in one of the frames of video; receiving temporal data specifying when the foreground object was observed in one of the frames of video; passing the set of kinematic data and the temporal data to an adaptive resonance theory (ART) network, wherein the ART network models observed behavior of a plurality of foreground objects observed in the scene, relative to the kinematic data and the temporal data; evaluating one or more clusters of the ART network to determine whether the set of kinematic data and temporal data passed to the ART network are indicative of an occurrence of a temporally anomaly; and upon determining a temporal anomaly has occurred, publishing an alert message. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. A system, comprising:
-
a video input source configured to provide a sequence of video frames, each depicting a scene; a processor; and a memory containing a program, which, when executed on the processor is configured to perform an operation for analyzing the scene, as depicted by the sequence of video frames captured by the video input source, the operation comprising; receiving a set of kinematic data derived by a computer vision engine observing a foreground object in one of the frames of video, receiving temporal data specifying when the foreground object was observed in one of the frames of video, passing the set of kinematic data and the temporal data to an adaptive resonance theory (ART) network, wherein the ART network models observed behavior of a plurality of foreground objects observed in the scene, relative to the kinematic data and the temporal data, evaluating one or more clusters of the ART network to determine whether the set of kinematic data and temporal data passed to the ART network are indicative of an occurrence of a temporally anomaly, and upon determining a temporal anomaly has occurred, publishing an alert message. - View Dependent Claims (21, 22, 23, 24, 25)
-
Specification