UNSUPERVISED LEARNING OF FEATURE ANOMALIES FOR A VIDEO SURVEILLANCE SYSTEM

• US 20140003710A1
• Filed: 06/27/2013
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: SEOW, Ming-Jung, COBB, Wesley Kenneth
• Granted Patent: US 9,111,148 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06N 3/0409 Adaptive resonance theory [ART] networks
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for analyzing a scene depicted in an input stream of video frames captured by a video camera. In one embodiment, e.g., a machine learning engine may include statistical engines for generating topological feature maps based on observations and a detection module for detecting feature anomalies. The statistical engines may include adaptive resonance theory (ART) networks which cluster observed position-feature characteristics. The statistical engines may further reinforce, decay, merge, and remove clusters. The detection module may calculate a rareness value relative to recurring observations and data in the ART networks. Further, the sensitivity of detection may be adjusted according to the relative importance of recently observed anomalies.

Claims

1. A computer-implemented method for analyzing a scene, the method comprising:

receiving kinematic and feature data for an object in the scene;

determining, via one or more processors, a position-feature vector from the received data, the position-feature vector representing a location and one or more feature values at the location;

retrieving a feature map corresponding to the position-feature vector, wherein the feature map includes one or more position-feature clusters;

determining a rareness value for the object based at least on the position feature vector and the feature map; and

reporting the object as anomalous if the rareness value meets given criteria.

2. The method of claim 1, further comprising, updating the feature map using the position-feature vector.

3. The method of claim 1, wherein the feature map includes one or more adaptive resonance theory (ART) network clusters, and wherein the rareness value is determined based on at least distance of the position-feature vector to a closest cluster and on statistical relevance of clusters less than a threshold distance from the position-feature vector.

4. The method of claim 3, wherein the distance to the closest cluster is a pseudo-Mahalanobis distance, and wherein distances to the clusters less than the threshold distance are determined as mean-squared error distances.

5. The method of claim 4, wherein the rareness value is determined as

6. The method of claim 3, wherein mean and variance of the clusters are actual mean and variance of input vectors which have mapped to the clusters, wherein each cluster is weighted by statistical relevance, and wherein the clusters are reinforced, decayed, merged, and removed by a statistical engine.

7. The method of claim 1, further comprising, modifying the feature map based on a frequency of observed anomalies before determining the rareness value.

8. The method of claim 7, wherein modifying the feature map includes increasing variances of clusters in the feature map based on counts of observed anomalies associated with each cluster.

9. The method claim 1, wherein the one or more appearance features include at least one of area, area derivative, shadow, shininess, internal energy, area, entropy, gradient histogram, hue, global positioning system (GPS) coordinates, and radio-frequency identification (RFID) tags.

10. The method of claim 1, wherein the kinematic and feature data are extracted from video frames captured by a video camera.

11. A non-transitory computer-readable storage medium storing instructions, which when executed by a computer system, perform operations for analyzing a scene, the operations comprising:

receiving kinematic and feature data for an object in the scene;

determining, via one or more processors, a position-feature vector from the received data, the position-feature vector representing a location and one or more feature values at the location;

retrieving a feature map corresponding to the position-feature vector, wherein the feature map includes one or more position-feature clusters;

determining a rareness value for the object based at least on the position feature vector and the feature map; and

reporting the object as anomalous if the rareness value meets given criteria.

12. The computer-readable storage medium of claim 11, the operations further comprising, updating the feature map using the position-feature vector.

13. The computer-readable storage medium of claim 11, wherein the feature map includes one or more adaptive resonance theory (ART) network clusters, and wherein the rareness value is determined based on at least distance of the position-feature vector to a closest cluster and on statistical relevance of clusters less than a threshold distance from the position-feature vector.

14. The computer-readable storage medium of claim 13, wherein the distance to the closest cluster is a pseudo-Mahalanobis distance, and wherein distances to the clusters less than the threshold distance are determined as mean-squared error distances.

15. The computer-readable storage medium of claim 14, wherein the rareness value is determined as

16. The computer-readable storage medium of claim 13, wherein mean and variance of the clusters are actual mean and variance of input vectors which have mapped to the clusters, wherein each cluster is weighted by statistical relevance, and wherein the clusters are reinforced, decayed, merged, and removed by a statistical engine.

17. The computer-readable storage medium of claim 11, the operations further comprising, modifying the feature map based on a frequency of observed anomalies before determining the rareness value.

18. The computer-readable storage medium of claim 17, wherein modifying the feature map includes increasing variances of clusters in the feature map based on counts of observed anomalies associated with each cluster.

19. The computer-readable storage medium of claim 11, wherein the one or more appearance features include at least one of area, area derivative, shadow, shininess, internal energy, area, entropy, gradient histogram, hue, global positioning system (GPS) coordinates, and radio-frequency identification (RFID) tags.

20. The computer-readable storage medium of claim 19, wherein the kinematic and feature data are extracted from video frames captured by a video camera.

21. A system, comprising:

a processor; and

a memory, wherein the memory includes an application program configured to perform operations for analyzing a scene, the operations comprising;

receiving kinematic and feature data for an object in the scene,

determining, via one or more processors, a position-feature vector from the received data, the position-feature vector representing a location and one or more feature values at the location,

retrieving a feature map corresponding to the position-feature vector, wherein the feature map includes one or more position-feature clusters,

determining a rareness value for the object based at least on the position feature vector and the feature map, and

reporting the object as anomalous if the rareness value meets given criteria.

22. The system of claim 21, wherein the feature map includes one or more adaptive resonance theory (ART) network clusters, and wherein the rareness value is determined based on at least distance of the position-feature vector to a closest cluster and on statistical relevance of clusters less than a threshold distance from the position-feature vector.

23. The system of claim 22, wherein the distance to the closest cluster is a pseudo-Mahalanobis distance, and wherein distances to the clusters less than the threshold distance are determined as mean-squared error distances.

25. The system of claim 22, wherein mean and variance of the clusters are actual mean and variance of input vectors which have mapped to the clusters, wherein each cluster is weighted by statistical relevance, and wherein the clusters are reinforced, decayed, merged, and removed by a statistical engine.

24. The system of claim 24, wherein the rareness value is determined as