ANOMALY DETECTION IN IMAGES AND VIDEOS

• US 20130101221A1
• Filed: 10/25/2011
• Published: 04/25/2013
• Est. Priority Date: 10/25/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Haas, Norman, Li, Ying, Otto, Charles A., Paluri, Balamanohar, Pankanti, Sharathchandra, Fujiki, Yuichi
• Granted Patent: US 8,724,904 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: B61L 23/044 Broken rails
• CPC Class Codes: B61L 23/047 Track or rail movements
• CPC Class Codes: B61L 23/048 Road bed changes, e.g. road bed erosion
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks

Abstract

A system, method, and computer program product for detecting anomalies in an image. In an example embodiment the method includes partitioning each image of a set of images into a plurality of image local units. The method further includes clustering all local units in the image set into clusters, and consequently assigning a class label to each local unit based on the clustering results. The local units with identical class labels having at least one substantially related image feature. Further, the method includes assigning a weight to each of the local units based on a variation of the class labels across all images in a set of images. The method further includes performing a clustering over all images in the set by using a distance metric that takes the learned weight of each local unit into account, then determining the images that belong to minorities of the clusters as anomalies.

Claims

1. A method for detecting anomalies in a new image, the method comprising:

partitioning each image of a set of images into a plurality of local units;

clustering all the local units in the set of images into clusters;

assigning class labels to the local units based on a clustering result from clustering the local units, the local units with identical class labels having at least one substantially related image feature;

assigning a weight to each of the local units based on a variation of the class labels across all images in the set of images; and

categorizing the new image as anomalous based on the weight assigned to the local units.

2. The method of claim 1, wherein the class labels indicate a semantic content of the local units.

3. The method of claim 1 further comprising:

clustering the set of images into a majority group and at least one minority group based on the weights of the local units;

determining the images belonging to the at least one minority group after clustering as anomalies;

categorizing the new image as an anomaly if the new image belongs to the at least one minority group.

4. The method of claim 3 wherein clustering the set of images progressively merges the set of images into a hierarchical tree.

5. The method of claim 3 wherein clustering the set of images is based, at least in part, on a distance-based clustering that determines a distance between images in the set of images by using the weights of the local units.

6. The method of claim 1 further comprising:

performing a global clustering to assign a semantic label for each local unit; and

determining a weight for each local unit by detecting a change of its semantic labels within a temporal window across all images in the set of images.

7. The method of claim 1 wherein the new image is a localized portion of a larger image.

8. The method of claim 1 further comprising extracting the substantially related image feature from each local unit.

9. The method of claim 8 wherein extracting the substantially related image feature from each local unit includes using at least one of Gabor, edge magnitude, and edge direction algorithms to extract the substantially related image feature.

10. The method of claim 1 further comprising:

calculating a centroid for each cluster of the local units; and

incrementally updating the centroid of each local unit cluster in real time by including the corresponding local units of the new image.

11. The method of claim 10, further comprising incrementally updating a weight of each of the local units in real-time by utilizing an original weight and an alternate weight array.

12. The method of claim 1, further comprising calculating the distances from the new image to centroids of clusters of the set of images.

13. The method of claim 1, further comprising:

assigning the new image to an image cluster that yields the shortest distance from the new image to the centroids of image clusters; and

determining if the new image is a normal image or an anomaly image based on the assigned image cluster.

14. The method of claim 1, further comprising incrementally updating centroids of image clusters after including the new image.

15. The method of claim 1, further comprising:

computing the distances from each of the local units of the new image to the centroids of a set of clusters created during clustering of the local units of the set of images;

assigning the local unit of the new image to the class label of the cluster that gives the shortest distance;

updating an alternate weight array with the shortest distance, the alternate weight array is the size of the number of local units in the new image; and

normalizing the alternate weight array after a plurality of iterations to calculate an updated weight array.

16. A computer program product for detecting tie plate anomalies in a tie plate image, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code configured to;

localize a tie plate region in an image from a set of tie plate images;

divide the tie plate region into a set of local units;

extract features from each local unit in the set of local units;

perform a clustering over all local units;

assign a class label to each local unit, the class label indicates a semantic content of the local unit;

determine the weight of each local unit across all images in the set of images; and

categorizing the tie plate image as anomalous based on the weight assigned to the local units.

17. The computer program product of claim 16, wherein the computer readable program code is further configured to represent each tie plate region from the set of images with concatenated unit labels, wherein the concatenated unit labels are feature vectors representing the tie plate region.

18. The computer program product of claim 16, wherein the computer readable program code is further configured to perform a global clustering over all tie plate regions in the set of images based on a distance calculation between images in the image set, wherein the distance calculation is based, in part, on the weight of each of the local units in each tie plate region.

19. The computer program product of claim 18, wherein the computer readable program code is further configured to determine the tie plate regions that belong to classes of minorities from the global clustering are anomalous.

20. A system for detecting anomalies in images, the system comprising:

a processor;

a memory coupled to the processor, the memory having computer readable program code embodied therewith, the computer readable program code configured to;

partition each image of a set of images into a plurality of image local units;

cluster all local units in the set of images into clusters;

assign class labels to the local units, the local units with identical class labels having at least one substantially related image feature; and

assign a weight to each of the local units based on a variation of the class labels across images in the set of images;

perform a clustering over all images in the set of images by using a distance metric that is based, at least in part, of the assigned weight of each of the local unit; and

declare the images that belong to the minorities of the clusters as anomalies.