EFFICIENT RETRIEVAL OF ANOMALOUS EVENTS WITH PRIORITY LEARNING

• US 20120294511A1
• Filed: 05/18/2011
• Published: 11/22/2012
• Est. Priority Date: 05/18/2011
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated (Kyndryl Holdings, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Datta, Ankur, Paluri, Balamanohar, Pankanti, Sharathchandra U., Zhai, Yun
• Granted Patent: US 9,158,976 B2
• Time in Patent Office: Days
• US Class Current: 382/155
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G16B 40/00 ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding

Abstract

Local models learned from anomaly detection are used to rank detected anomalies. The local models include image feature values extracted from an image field of video image data with respect to different predefined spatial and temporal local units, wherein anomaly results are determined by failures to fit to applied anomaly detection module local models. Image features values extracted from the image field local units associated with anomaly results are normalized, and image feature values extracted from the image field local units are clustered. Weights for anomaly results are learned as a function of the relations of the normalized extracted image feature values to the clustered image feature values. The normalized values are multiplied by the learned weights to generate ranking values to rank the anomalies.

Claims

1. A method for using models learned from anomaly detection to rank detected anomalies, the method comprising:

retrieving a plurality of anomaly results from an anomaly detection module comprising a plurality of local models in response to an input query for an anomaly, wherein the anomaly detection module local models comprise image feature values extracted from an image field of video image data with respect to each of a plurality of different predefined spatial and temporal local units, and each of the plurality of anomaly results are determined by respective failures to fit to applied ones of the anomaly detection module local models;

normalizing each of a plurality of values of image features extracted from the image field local units and that are associated with each of the plurality of anomaly results;

clustering image feature values extracted from the each image field local units that are associated with the each of the plurality of anomaly results;

learning each of a plurality of weights for each of the anomaly results as a function of a relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field local units;

multiplying the normalized values of extracted features of each of the plurality of anomalies by their respective learned weights to generate respective ranking values; and

ranking the plurality of anomalies by their generated respective ranking values.

2. The method of claim 1, wherein the step of learning each of the plurality of weights for each of the anomaly results as a function of the relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field local units further comprises:

determining spatial locations of each of the retrieved anomalies within the field of view of the input video data as correlated to features of interest of a real-world scene represented within the field of view; and

assigning a first weighting to a first of the retrieved anomalies that is higher than a second weighting assigned to a second of the retrieved anomalies in response to the determined spatial location of the first anomaly being within a portion of the field of view of the input video that is correlated with a cordoned off area of the real-world scene and the determined spatial location of the second anomaly being outside the portion; and

wherein the normalized extracted features of each of the first and second anomalies are outliers from and have the same distance to a center of a same learned model cluster of extracted features of a one of the anomaly detection module local models.

3. The method of claim 2, further comprising:

updating the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in relative distances of respective normalized extracted features from a center of a same associated learned model cluster of the extracted features.

4. The method of claim 3, further comprising:

updating the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in temporal data of the respective first and second anomalies.

5. The method of claim 4, wherein the difference in the temporal data is a longer duration time of an associated tracked object movement.

6. The method of claim 4, wherein the difference in the temporal data indicates a difference in a time of day of occurrences of the respective first and second anomalies.

7. The method of claim 4, further comprising:

assigning a prior importance value to a camera associated with one of the first and second anomalies based on a type of anomaly expected to occur in video data from the camera;

generating an updated importance value for the camera based on at least one of a type of observed anomaly or a number of observed anomalies that occur in the video data from the camera as determined by anomaly detection; and

updating the weighting of the one of the first and second anomalies associated with the camera by increasing or decreasing the weighting relative to the weighting of the other of the first and second anomalies as a function of a product of the prior importance value multiplied by the updated importance value.

8. A system, comprising:

a processing unit, computer readable memory and a computer readable storage medium;

first program instructions to retrieve a plurality of anomaly results from an anomaly detection module comprising a plurality of local models in response to an input query for an anomaly, wherein the anomaly detection module local models comprise image feature values extracted from an image field of video image data with respect to each of a plurality of different predefined spatial and temporal local units, and each of the plurality of anomaly results are determined by respective failures to fit to applied ones of the anomaly detection module local models;

second program instructions to normalize each of a plurality of values of image features extracted from the image field local units and that are associated with each of the plurality of anomaly results;

third program instructions to cluster image feature values extracted from the each image field local units that are associated with the each of the plurality of anomaly results, and to learn each of a plurality of weights for each of the anomaly results as a function of a relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field local units; and

fourth program instructions to multiply the normalized values of extracted features of each of the plurality of anomalies by their respective learned weights to generate respective ranking values and rank the plurality of anomalies by their generated respective ranking values; and

wherein the first, second, third and fourth program instructions are stored on the computer readable storage medium for execution by the processing unit via the computer readable memory.

9. The system of claim 8, wherein the third program instructions are further to learn each of the plurality of weights for each of the anomaly results as a function of the relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field local units by:

determining spatial locations of each of the retrieved anomalies within the field of view of the input video data as correlated to features of interest of a real-world scene represented within the field of view; and

assigning a first weighting to a first of the retrieved anomalies that is higher than a second weighting assigned to a second of the retrieved anomalies in response to the determined spatial location of the first anomaly being within a portion of the field of view of the input video that is correlated with a cordoned off area of the real-world scene and the determined spatial location of the second anomaly being outside the portion; and

wherein the normalized extracted features of each of the first and second anomalies are outliers from and have the same distance to a center of a same learned model cluster of extracted features of a one of the anomaly detection module local models.

10. The system of claim 9, wherein the third program instructions are further to update the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in relative distances of respective normalized extracted features from a center of a same associated learned model cluster of the extracted features.

11. The system of claim 10, wherein the third program instructions are further to update the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in temporal data of the respective first and second anomalies.

12. The system of claim 11, wherein the third program instructions are further to:

assign a prior importance value to a camera associated with one of the first and second anomalies based on a type of anomaly expected to occur in video data from the camera;

generate an updated importance value for the camera based on at least one of a type of observed anomaly or a number of observed anomalies that occur in the video data from the camera as determined by anomaly detection; and

update the weighting of the one of the first and second anomalies associated with the camera by increasing or decreasing the weighting relative to the weighting of the other of the first and second anomalies as a function of a product of the prior importance value multiplied by the updated importance value.

13. An article of manufacture, comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising instructions that, when executed by a computer processor, cause the computer processor to;

retrieve a plurality of anomaly results from an anomaly detection module comprising a plurality of local models in response to an input query for an anomaly, wherein the anomaly detection module local models comprise image feature values extracted from an image field of video image data with respect to each of a plurality of different predefined spatial and temporal local units, and each of the plurality of anomaly results are determined by respective failures to fit to applied ones of the anomaly detection module local models;

normalize each of a plurality of values of image features extracted from the image field local units and that are associated with each of the plurality of anomaly results;

cluster image feature values extracted from the each image field local units that are associated with the each of the plurality of anomaly results;

learn each of a plurality of weights for each of the anomaly results as a function of a relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field local units;

multiply the normalized values of extracted features of each of the plurality of anomalies by their respective learned weights to generate respective ranking values; and

rank the plurality of anomalies by their generated respective ranking values.

14. The article of manufacture of claim 13, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to learn each of the plurality of weights for each of the anomaly results as a function of the relation of their normalized values of the extracted image features to the clustered image feature values extracted from the each respective associated image field local units by:

determining spatial locations of each of the retrieved anomalies within the field of view of the input video data as correlated to features of interest of a real-world scene represented within the field of view; and

assigning a first weighting to a first of the retrieved anomalies that is higher than a second weighting assigned to a second of the retrieved anomalies in response to the determined spatial location of the first anomaly being within a portion of the field of view of the input video that is correlated with a cordoned off area of the real-world scene and the determined spatial location of the second anomaly being outside the portion; and

wherein the normalized extracted features of each of the first and second anomalies are outliers from and have the same distance to a center of a same learned model cluster of extracted features of a one of the anomaly detection module local models.

15. The article of manufacture of claim 14, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to update the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in relative distances of respective normalized extracted features from a center of a same associated learned model cluster of the extracted features.

16. The article of manufacture of claim 15, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to update the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in temporal data of the respective first and second anomalies.

17. The article of manufacture of claim 16, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to:

assign a prior importance value to a camera associated with one of the first and second anomalies based on a type of anomaly expected to occur in video data from the camera;

generate an updated importance value for the camera based on at least one of a type of observed anomaly or a number of observed anomalies that occur in the video data from the camera as determined by anomaly detection; and

update the weighting of the one of the first and second anomalies associated with the camera by increasing or decreasing the weighting relative to the weighting of the other of the first and second anomalies as a function of a product of the prior importance value multiplied by the updated importance value.

19. The method of claim 13, wherein the priority learning component learns each of the plurality of weights for each of the anomaly results as a function of the relation of their normalized values of the extracted image features to the clustered image feature values extracted from the respective associated each image field local units by:

determining spatial locations of each of the retrieved anomalies within the field of view of the input video data as correlated to features of interest of a real-world scene represented within the field of view; and

assigning a first weighting to a first of the retrieved anomalies that is higher than a second weighting assigned to a second of the retrieved anomalies in response to the determined spatial location of the first anomaly being within a portion of the field of view of the input video that is correlated with a cordoned off area of the real-world scene and the determined spatial location of the second anomaly being outside the portion; and

wherein the normalized extracted features of each of the first and second anomalies are outliers from and have the same distance to a center of a same learned model cluster of extracted features of a one of the anomaly detection module local models.

20. The method of claim 19, wherein the priority learning component updates the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in relative distances of respective normalized extracted features from a center of a same associated learned model cluster of the extracted features.

21. The method of claim 20, wherein the priority learning component updates the weighting of one of the first and second anomalies by increasing or decreasing the updated weighting relative to the weighting of the other of the first and second anomalies as a function of a difference in temporal data of the respective first and second anomalies.

22. The method of claim 21, wherein the priority learning component further:

assigns a prior importance value to a camera associated with one of the first and second anomalies based on a type of anomaly expected to occur in video data from the camera;

generates an updated importance value for the camera based on at least one of a type of observed anomaly or a number of observed anomalies that occur in the video data from the camera as determined by anomaly detection; and

updates the weighting of the one of the first and second anomalies associated with the camera by increasing or decreasing the weighting relative to the weighting of the other of the first and second anomalies as a function of a product of the prior importance value multiplied by the updated importance value.

18. A method for providing a service for using models learned from anomaly detection to rank detected anomalies, the method comprising:

providing a results retriever that retrieves a plurality of anomaly results from an anomaly detection module comprising a plurality of local models in response to an input query for an anomaly, wherein the anomaly detection module local models comprise image feature values extracted from an image field of video image data with respect to each of a plurality of different predefined spatial and temporal local units, and each of the plurality of anomaly results are determined by respective failures to fit to applied ones of the anomaly detection module local models;

providing a priority learning component that normalizes each of a plurality of values of image features extracted from the image field local units and that are associated with each of the plurality of anomaly results, clusters image feature values extracted from the each image field local units that are associated with the each of the plurality of anomaly results, learns each of a plurality of weights for each of the anomaly results as a function of a relation of their normalized values of the extracted image features to the clustered image feature values extracted from the respective associated each image field local units; and

provides a ranker that multiplies the normalized values of extracted features of each of the plurality of anomalies by their respective learned weights to generate respective ranking values and ranks the plurality of anomalies by their generated respective ranking values.