IDENTIFYING ANOMALOUS OBJECT TYPES DURING CLASSIFICATION

• US 20110052068A1
• Filed: 08/31/2009
• Published: 03/03/2011
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: FRIEDLANDER, David, SEOW, MING-JUNG, XU, GANG, GOTTUMUKKAL, RAJKIRAN KUMAR, COBB, WESLEY KENNETH
• Granted Patent: US 8,270,733 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for identifying anomaly object types during classification of foreground objects extracted from image data. A self-organizing map and adaptive resonance theory (SOM-ART) network is used to discover object type clusters and classify objects depicted in the image data based on pixel-level micro-features that are extracted from the image data. Importantly, the discovery of the object type clusters is unsupervised, i.e., performed independent of any training data that defines particular objects, allowing a behavior-recognition system to forgo a training phase and for object classification to proceed without being constrained by specific object definitions. The SOM-ART network is adaptive and able to learn while discovering the object type clusters and classifying objects and identifying anomaly object types.

Claims

1. A computer-implemented method for identifying anomaly object types during classification of image data captured by a video camera, the method comprising:

receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object within the image data;

classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters based on the micro-feature vector;

computing a probability density function for the object type clusters;

computing a probability density value for the micro-feature vector;

evaluating a rareness measure of the micro-feature vector, wherein the rareness measure estimates a likelihood of observing the micro-feature vector, based on the probability density function and the probability density value; and

identifying the foreground object as an anomaly object type when the rareness measure is below a specified threshold.

2. The computer-implemented method of claim 1, wherein the foreground object is identified as the anomaly object type when the rareness measure is greater than a threshold value.

3. The computer-implemented method of claim 1, wherein the foreground object is classified as depicting the first object type when a distance between the micro-feature vector and a center of the first object type cluster is less than a threshold value.

4. The computer-implemented method of claim 1, further comprising processing the micro-feature vector by a self-organizing map adaptive resonance theory (SOM-ART) network to discover the object type clusters for the image data.

5. The computer-implemented method of claim 1, further comprising merging the first object type cluster with a second object type cluster of the object type clusters for the image data when the first object type cluster overlaps the second object type cluster by a specified amount.

6. The computer-implemented method of claim 1, further comprising:

receiving additional micro-feature vectors; and

removing the first object type cluster when none of the additional micro-feature vectors match the first object type cluster.

7. The computer-implemented method of claim 1, further comprising adding a new object type cluster when the micro-feature vector does not correspond to any of the object type clusters for the image data.

8. The computer-implemented method of claim 1, further comprising:

receiving additional micro-feature vectors; and

classifying additional foreground objects based on the additional micro-feature vectors before evaluating the rareness measure of the micro-feature vector.

9. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for identifying anomaly object types during classification of image data captured by a video camera, the operation comprising:

receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object within the image data;

classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters based on the micro-feature vector;

computing a probability density function for the object type clusters;

computing a probability density value for the micro-feature vector;

evaluating a rareness measure of the micro-feature vector, wherein the rareness measure estimates a likelihood of observing the micro-feature vector, based on the probability density function and the probability density value; and

identifying the foreground object as an anomaly object type when the rareness measure is below a specified threshold.

10. The computer-readable storage medium of claim 9, wherein the foreground object is identified as the anomaly object type when the rareness measure is greater than a threshold value.

11. The computer-readable storage medium of claim 9, wherein the foreground object is classified as depicting the first object type when a distance between the micro-feature vector and a center of the first object type cluster is less than a threshold value.

12. The computer-readable storage medium of claim 9, further comprising processing the micro-feature vector by a self-organizing map adaptive resonance theory (SOM-ART) network to discover the object type clusters for the image data.

13. The computer-readable storage medium of claim 9, further comprising:

receiving additional micro-feature vectors; and

classifying additional foreground objects based on the additional micro-feature vectors before evaluating the rareness measure of the micro-feature vector.

14. A system, comprising:

a video input source configured to provide image data;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation that identifies anomaly object types during classification of image data captured by a video camera, the operation comprising;

receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object within the image data;

classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters based on the micro-feature vector;

computing a probability density function for the object type clusters;

computing a probability density value for the micro-feature vector;

evaluating a rareness measure of the micro-feature vector, wherein the rareness measure estimates a likelihood of observing the micro-feature vector, based on the probability density function and the probability density value, and

identifying the foreground object as an anomaly object type when the rareness measure is below a specified threshold.

15. The system of claim 14, wherein the foreground object is identified as the anomaly object type when the rareness measure is greater than a threshold value.

16. The system of claim 14, wherein the foreground object is classified as depicting the first object type when a distance between the micro-feature vector and a center of the first object type cluster is less than a threshold value.

17. The system of claim 14, further comprising processing the micro-feature vector by a self-organizing map adaptive resonance theory (SOM-ART) network to discover the object type clusters for the image data.

18. The system of claim 14, further comprising merging the first object type cluster with a second object type cluster of the object type clusters for the image data when the first object type cluster overlaps the second object type cluster by a specified amount.

19. The system of claim 14, further comprising:

receiving additional micro-feature vectors; and

removing the first object type cluster when none of the additional micro-feature vectors match the first object type cluster.

20. The system of claim 14, further comprising adding a new object type cluster when the micro-feature vector does not correspond to any of the object type clusters for the image data.