Clustering nodes in a self-organizing map using an adaptive resonance theory network

• US 8,270,732 B2
• Filed: 08/31/2009
• Issued: 09/18/2012
• Est. Priority Date: 08/31/2009
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Behavioral Recognition Systems Incorporated
• Inventors: Cobb, Wesley Kenneth, Friedlander, David, Saitwal, Kishor Adinath, Seow, Ming-Jung, Xu, Gang
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US12/551,154
• Publication Number:
  US 20110052067A1
• Time in Patent Office: 1,114 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/155, 382/168, 382/173, 382/181, 382/193, 382/199, 382/209, 382/218, 382/219, 382/224, 382/232, 382/254, 382/274, 382/276, 382/282, 382286-294, 382/305, 382/312, 340/573.1, 340/948, 707/791
• US Class Current: 382/224
• CPC Class Codes: G06F 18/23211 with adaptive number of clusters
• CPC Class Codes: G06N 3/0409 Adaptive resonance theory [ART] networks
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Techniques are disclosed for discovering object type clusters using pixel-level micro-features extracted from image data. A self-organizing map and adaptive resonance theory (SOM-ART) network is used to classify objects depicted in the image data based on the pixel-level micro-features. Importantly, the discovery of the object type clusters is unsupervised, i.e., performed independent of any training data that defines particular objects, allowing a behavior-recognition system to forgo a training phase and for object classification to proceed without being constrained by specific object definitions. The SOM-ART network is adaptive and able to learn while discovering the object type clusters and classifying objects.

Claims

1. A computer-implemented method for discovering object type clusters for image data captured by a video camera, the method comprising:

receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object;

processing the micro-feature vector by a self-organizing map adaptive resonance theory (SOM-ART) network to discover the object type clusters for the image data;

classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters when the micro-feature vector matches the first object type cluster; and

indicating that no match is found when the micro-feature vector does not correspond to any of the object type clusters for the image data.

2. The computer-implemented method of claim 1, further comprising:

receiving additional micro-feature vectors; and

updating the SOM-ART network when a predetermined number of the additional micro-feature vectors is received to produce an updated SOM-ART network.

3. The computer-implemented method of claim 2, further comprising:

determining that a micro-feature value is invalid; and

excluding the invalid micro-feature value from the updating of the SOM-ART network.

4. The computer-implemented method of claim 1, further comprising estimating a topology for the SOM-ART network based on the number of input micro-feature vectors.

5. The computer-implemented method of claim 1, further comprising merging the first object type cluster with a second object type cluster of the object type clusters for the image data when the first object type cluster overlaps the second object type cluster by a specified amount.

6. The computer-implemented method of claim 1, further comprising:

receiving additional micro-feature vectors; and

removing the first object type cluster when none of the additional micro-feature vectors match the first object type cluster.

7. The computer-implemented method of claim 1, further comprising adding a new object type cluster when the micro-feature vector does not correspond to any of the object type clusters for the image data.

8. The computer-implemented method of claim 1, further comprising:

receiving additional micro-feature vectors; and

processing the additional micro-feature vectors to produce a mature SOM-ART network before classifying the foreground patch.

9. A computer-readable storage medium containing a program which, when executed by a processor, performs an operation for discovering object type clusters for image data captured by a video camera, the operation comprising:

receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object;

processing the micro-feature vector by a self-organizing map adaptive resonance theory (SOM-ART) network to discover the object type clusters for the image data;

classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters when the micro-feature vector matches the first object type cluster; and

indicating that no match is found when the micro-feature vector does not correspond to any of the object type clusters for the image data.

10. The computer-readable storage medium of claim 9, wherein the operation further comprises:

receiving additional micro-feature vectors; and

updating the SOM-ART network when a predetermined number of the additional micro-feature vectors is received to produce an updated SOM-ART network.

11. The computer-readable storage medium of claim 10, wherein the operation further comprises:

determining that a micro-feature value is invalid; and

excluding the invalid micro-feature value from the updating of the SOM-ART network.

12. The computer-readable storage medium of claim 9, wherein the operation further comprises estimating a topology for the SOM-ART network based on the number of input micro-feature vectors.

13. The computer-readable storage medium of claim 9, wherein the operation further comprises:

receiving additional micro-feature vectors; and

processing the additional micro-feature vectors to produce a mature SOM-ART network before classifying the foreground patch.

14. A system, comprising:

a video input source configured to provide image data;

a processor; and

a memory containing a program, which, when executed on the processor is configured to perform an operation discovering object type clusters for the image data captured by the video input source, the operation comprising;

receiving a micro-feature vector including multiple micro-feature values, each micro-feature value based on at least one pixel-level characteristic of a foreground patch that depicts a foreground object;

processing the micro-feature vector by a self-organizing map adaptive resonance theory (SOM-ART) network to discover the object type clusters for the image data;

classifying the foreground object as depicting a first object type corresponding to a first object type cluster of the object type clusters when the micro-feature vector matches the first object type cluster; and

indicating that no match is found when the micro-feature vector does not correspond to any of the object type clusters for the image data.

15. The system of claim 14, wherein the operation further comprises:

receiving additional micro-feature vectors; and

updating the SOM-ART network when a predetermined number of the additional micro-feature vectors is received to produce an updated SOM-ART network.

16. The system of claim 15, wherein the operation further comprises:

determining that a micro-feature value is invalid; and

excluding the invalid micro-feature value from the updating of the SOM-ART network.

17. The system of claim 14, wherein the operation further comprises comprises estimating a topology for the SOM-ART network based on the number of input micro-feature vectors.

18. The system of claim 14, wherein the operation further comprises:

receiving additional micro-feature vectors; and

processing the additional micro-feature vectors to produce a mature SOM-ART network before classifying the foreground patch.

19. The system of claim 14, wherein the operation further comprises:

receiving additional micro-feature vectors; and

removing the first object type cluster when none of the additional micro-feature vectors match the first object type cluster.

20. The system of claim 14, wherein the operation further comprises adding a new object type cluster when the micro-feature vector does not correspond to any of the object type clusters for the image data.