IDENTIFYING ANOMALOUS OBJECT TYPES DURING CLASSIFICATION

• US 20130022242A1
• Filed: 09/18/2012
• Published: 01/24/2013
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Xu, Gang, FRIEDLANDER, David, Seow, Ming-Jung, Gottumukkal, Rajkiran Kumar
• Granted Patent: US 8,548,198 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for identifying anomaly object types during classification of foreground objects extracted from image data. A self-organizing map and adaptive resonance theory (SOM-ART) network is used to discover object type clusters and classify objects depicted in the image data based on pixel-level micro-features that are extracted from the image data. Importantly, the discovery of the object type clusters is unsupervised, i.e., performed independent of any training data that defines particular objects, allowing a behavior-recognition system to forgo a training phase and for object classification to proceed without being constrained by specific object definitions. The SOM-ART network is adaptive and able to learn while discovering the object type clusters and classifying objects and identifying anomaly object types.

Claims

1. A computer-implemented method for characterizing one or more objects depicted in frames of video captured by a video camera, the method comprising:

for each successive video frame;

identifying one or more foreground objects depicted in the video frame, and for each foreground object;

determining one or more appearance characteristics of the foreground object from pixels of the video frame depicting the foreground object, and

determining one or more kinematic characteristics of the foreground object from the pixels of the video frame depicting the foreground object; and

clustering the determined appearance characteristics and the determined kinematic characteristics of each of the foreground objects to determine one or more object type clusters for classifying objects depicted in the frames of video.

2. The method of claim 1, further comprising, generating, for each identified foreground object, for each frame, a micro-feature vector from the determined appearance characteristics and the determined kinematic characteristics of the foreground object.

3. The method of claim 2, wherein clustering the determined appearance characteristics and the determined kinematic characteristics of each of the foreground objects comprises processing each of the micro-feature vectors by a self-organizing map adaptive resonance theory (SOM-ART) network.

4. The method of claim 1, further comprising:

determining that at least a first object type cluster has matured; and

classifying a first foreground object depicted in a subsequent video frame captured by the video camera as depicting a first object type corresponding to the first object type cluster.

5. The method of claim 1, wherein at least one of the appearance characteristics is derived from a color characteristics of the pixels of the video frame depicting the foreground object.

6. The method of claim 1, wherein at least one of the kinematic characteristics corresponds to a coordinate position of the foreground object in the frame of video.

7. The method of claim 1, wherein at least one of the kinematic characteristics corresponds to either an acceleration or velocity of the foreground object over a plurality of the video frames.

8. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for characterizing one or more objects depicted in frames of video captured by a video camera, the operation comprising:

for each successive video frame;

identifying one or more foreground objects depicted in the video frame, and for each foreground object;

determining one or more appearance characteristics of the foreground object from pixels of the video frame depicting the foreground object, and

determining one or more kinematic characteristics of the foreground object from the pixels of the video frame depicting the foreground object; and

clustering the determined appearance characteristics and the determined kinematic characteristics of each of the foreground objects to determine one or more object type clusters for classifying objects depicted in the frames of video.

9. The computer-readable storage medium of claim 8, wherein the operation further comprises, generating, for each identified foreground object, for each frame, a micro-feature vector from the determined appearance characteristics and the determined kinematic characteristics of the foreground object.

10. The computer-readable storage medium of claim 9, wherein clustering the determined appearance characteristics and the determined kinematic characteristics of each of the foreground objects comprises processing each of the micro-feature vectors by a self-organizing map adaptive resonance theory (SOM-ART) network.

11. The computer-readable storage medium of claim 8, wherein the operation further comprises:

determining that at least a first object type cluster has matured; and

classifying a first foreground object depicted in a subsequent video frame captured by the video camera as depicting a first object type corresponding to the first object type cluster.

12. The computer-readable storage medium of claim 8, wherein at least one of the appearance characteristics is derived from a color characteristics of the pixels of the video frame depicting the foreground object.

13. The computer-readable storage medium of claim 8, wherein at least one of the kinematic characteristics corresponds to a coordinate position of the foreground object in the frame of video.

14. The computer-readable storage medium of claim 8, wherein at least one of the kinematic characteristics corresponds to either an acceleration or velocity of the foreground object over a plurality of the video frames.

15. A system, comprising:

a video input source configured to provide image data;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation for characterizing one or more objects depicted in frames of video captured by a video camera, the operation comprising;

for each successive video frame;

identifying one or more foreground objects depicted in the video frame, and for each foreground object;

determining one or more appearance characteristics of the foreground object from pixels of the video frame depicting the foreground object, and

determining one or more kinematic characteristics of the foreground object from the pixels of the video frame depicting the foreground object; and

clustering the determined appearance characteristics and the determined kinematic characteristics of each of the foreground objects to determine one or more object type clusters for classifying objects depicted in the frames of video.

16. The system of claim 15, wherein the operation further comprises, generating, for each identified foreground object, for each frame, a micro-feature vector from the determined appearance characteristics and the determined kinematic characteristics of the foreground object.

17. The system of claim 16, wherein clustering the determined appearance characteristics and the determined kinematic characteristics of each of the foreground objects comprises processing each of the micro-feature vectors by a self-organizing map adaptive resonance theory (SOM-ART) network.

18. The system of claim 15, wherein the operation further comprises:

determining that at least a first object type cluster has matured; and

classifying a first foreground object depicted in a subsequent video frame captured by the video camera as depicting a first object type corresponding to the first object type cluster.

19. The system of claim 15, wherein at least one of the appearance characteristics is derived from a color characteristics of the pixels of the video frame depicting the foreground object.

20. The system of claim 15, wherein at least one of the kinematic characteristics corresponds to a coordinate position of the foreground object in the frame of video.

21. The system of claim 15, wherein at least one of the kinematic characteristics corresponds to either an acceleration or velocity of the foreground object over a plurality of the video frames.