SYSTEMS AND METHODS FOR DATA-DRIVEN ANOMALY DETECTION

• US 20140379301A1
• Filed: 03/19/2014
• Published: 12/25/2014
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Infosys Limited
• Original Assignee: Infosys Limited
• Inventors: Shastri, Lokendra, Raj, K. Antony Arokia Durai, Kanagasabapathi, Balasubramanian
• Granted Patent: US 10,552,511 B2
• Time in Patent Office: Days
• US Class Current: 702/183
• CPC Class Codes: G05B 23/0224 Process history based detection method, e.g. whereby history implies the availability of large amounts of data
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection

Abstract

The technique relates to a system and method for data-driven anomaly detection. This technique involves identifying region of interest from the data based on dimensionality reduction technique and change point detection algorithm. A reference data can be obtained separately or can be obtained from the test data also, wherein the reference data represent the normal operating condition of a system. The reference data are classified into different groups representing different modes of operation of the system. A control limit is determined for the different groups. The data within the region of interest are mapped with the different groups of the reference data and it is determined if the mapped data fall outside of the control limit of the mapped group. Finally, at least one abnormal event is detected by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

Claims

1. A computer-implemented method for data-driven anomaly detection, the said method comprising:

identifying, by a processor, a region of interest from the data based on a dimensionality reduction technique and a change point detection algorithm;

mapping, by the processor, the data within the region of interest with one or more predefined groups of reference data representing one or more modes of operation of a system, wherein the reference data represent normal operating condition of the system;

determining, by the processor, whether the data within the region of interest is outside of a predefined control limit of the corresponding mapped group of the one or more predefined groups; and

detecting, by the processor, at least one abnormal event by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

2. The method as claimed in claim 1 further comprising:

determining an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

3. The method as claimed in claim 1, wherein the data is captured from one or more sensors.

4. The method as claimed in claim 3, wherein at least one of the one or more sensors is identified which corresponds to the data within the region of interest and falls outside of the control limit by applying the heuristic algorithm.

5. The method as claimed in claim 1, wherein the dimensionality reduction technique includes T2 statistic.

6. The method as claimed in claim 1, wherein the one or more groups are classified based on one or more clustering algorithms.

7. The method as claimed in claim 1 further comprising:

generating an alert on the occurrence of the at least one abnormal event.

8. A computer-implemented method for data-driven anomaly detection, the said method comprising:

identifying, by a processor, a region of interest from the data based on a dimensionality reduction technique and a change point detection algorithm;

classifying, by the processor, reference data into one or more groups representing one or more modes of operation of a system, wherein the reference data is obtained from a region outside the region of interest;

determining, by the processor, a control limit for each of the one or more groups by analyzing the reference data;

mapping, by the processor, the data within the region of interest with the one or more groups;

determining, by the processor, whether the data within the region of interest is outside the control limit of the mapped group of the one or more predefined groups; and

detecting, by the processor, at least one abnormal event by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

9. The method as claimed in claim 8, further comprising:

detecting an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

10. The method as claimed in claim 8, wherein the data is captured from one or more sensors.

11. The method as claimed in claim 10, wherein at least one of the one or more sensors is identified which corresponds to the data within the region of interest and falls outside of the control limit by applying the heuristic algorithm.

12. The method as claimed in claim 8, wherein the dimensionality reduction technique includes T2 statistic.

13. The method as claimed in claim 8, wherein the one or more groups are classified based on one or more clustering algorithms.

14. The method as claimed in claim 8 further comprising:

generating an alert on the occurrence of the at least one abnormal event.

15. A system for data-driven anomaly detection, comprising:

a processor in operable communication with a processor-readable storage medium, the processor-readable storage medium containing one or more programming instructions whereby the processor is configured to implement;

a region of interest identification module configured to identify a region of interest from the data based on dimensionality reduction technique and change point detection algorithm;

a mapping module configured to map the data within the region of interest with one or more predefined groups of reference data representing one or more modes of operation of a system, wherein the reference data represent normal operating condition of the system;

a data analysis module configured to determine whether the data within the region of interest is outside of a predefined control limit of the corresponding mapped group of the one or more predefined groups; and

an abnormal event detection module configured to detect at least one abnormal event by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

16. The system as claimed in claim 15 further comprising:

an earliest time detection module configured to determine an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

17. The system as claimed in claim 15, wherein the data is captured from one or more sensors.

18. The system as claimed in claim 17, wherein at least one of the one or more sensors is identified which corresponds to the data within the region of interest and falls outside of the control limit by applying the heuristic algorithm.

19. The system as claimed in claim 15, wherein the dimensionality reduction technique includes T2 statistic.

20. The system as claimed in claim 15, wherein, the one or more groups are classified based on one or more clustering algorithms

21. The system as claimed in claim 15 further comprising:

an alert generation module configured to generate an alert on the occurrence of the at least one abnormal event.

22. A system for data-driven anomaly detection, comprising:

a processor in operable communication with a processor-readable storage medium, the processor-readable storage medium containing one or more programming instructions whereby the processor is configured to implement;

a region of interest identification module configured to identify a region of interest from the data based on dimensionality reduction technique and change point detection algorithm;

a reference data classification module configured to classify reference data into one or more groups representing one or more modes of operation of a system, wherein the reference data is obtained from a region outside the region of interest;

a control limit determination module configured to determine a control limit for each of the one or more groups by analyzing the reference data;

a mapping module configured to map the data within the region of interest with the one or more groups;

a data analysis module configured to determine whether the data within the region of interest is outside the control limit of the mapped group of the one or more predefined groups; and

an abnormal event detection module configured to detect at least one abnormal event by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

23. The system as claimed in claim 22 further comprising:

an earliest time detection module configured to determine an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

24. The system as claimed in claim 22, wherein the data is captured from one or more sensors.

25. The system as claimed in claim 24, wherein at least one of the one or more sensors is identified which corresponds to the data within the region of interest and falls outside of the control limit by applying the heuristic algorithm.

26. The system as claimed in claim 22, wherein the dimensionality reduction technique includes T2 statistic.

27. The system as claimed in claim 22 further comprising:

an alert generation module configured to generate an alert on the occurrence of the at least one abnormal event.

28. A computer-readable storage medium, that is not a signal, having computer-executable instructions stored thereon for data-driven anomaly detection, the said instructions comprising:

instructions for identifying a region of interest from the data based on dimensionality reduction technique and change point detection algorithm;

instructions for mapping the data within the region of interest with one or more predefined groups of reference data representing one or more modes of operation of a system, wherein the reference data represent normal operating condition of the system;

instructions for determining whether the data within the region of interest is outside of a predefined control limit of the corresponding mapped group of the one or more predefined groups; and

instructions for detecting at least one abnormal event by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

29. The computer-readable storage medium as claimed in claim 28 further comprising:

instructions for determining an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

30. The computer-readable storage medium as claimed in claim 28 further comprising:

instructions for generating an alert on the occurrence of the at least one abnormal event.

31. A computer-readable storage medium, that is not a signal, having computer executable instructions stored thereon for data-driven anomaly detection, the said instructions comprising:

instructions for identifying a region of interest from the data based on dimensionality reduction technique and change point detection algorithm;

instructions for classifying reference data into one or more groups representing one or more modes of operation of a system, wherein the reference data is obtained from a region outside the region of interest;

instructions for determining a control limit for each of the one or more groups by analyzing the reference data;

instructions for mapping the data within the region of interest with the one or more groups;

instructions for determining whether the data within the region of interest is outside the control limit of the mapped group of the one or more predefined groups; and

instructions for detecting at least one abnormal event by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

32. The computer-readable storage medium as claimed in claim 31 further comprising:

instructions for detecting an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

33. The computer-readable storage medium as claimed in claim 31 further comprising:

instructions for generating an alert on the occurrence of the at least one abnormal event.