Systems and methods for data-driven anomaly detection

• US 10,552,511 B2
• Filed: 03/19/2014
• Issued: 02/04/2020
• Est. Priority Date: 06/24/2013
• Status: Active Grant

• Current Assignee: Infosys Limited
• Original Assignee: Infosys Limited
• Inventors: Shastri, Lokendra, Raj, K. Antony Arokia Durai, Kanagasabapathi, Balasubramanian
• Primary Examiner(s): Charioui, Mohamed
• Assistant Examiner(s): Rastovski, Catherine T.
• Application Number:
  US14/220,050
• Publication Number:
  US 20140379301A1
• Time in Patent Office: 2,148 Days
• Field of Search: 702183, 382103, 382124, 382128, 382195, 382242
• CPC Class Codes: G05B 23/0224 Process history based detection method, e.g. whereby history implies the availability of large amounts of data
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection

Abstract

The technique relates to a system and method for data-driven anomaly detection. This technique involves identifying region of interest from the data based on dimensionality reduction technique and change point detection algorithm. A reference data can be obtained separately or can be obtained from the test data also, wherein the reference data represent the normal operating condition of a system. The reference data are classified into different groups representing different modes of operation of the system. A control limit is determined for the different groups. The data within the region of interest are mapped with the different groups of the reference data and it is determined if the mapped data fall outside of the control limit of the mapped group. Finally, at least one abnormal event is detected by applying a heuristic algorithm on the data within the region of interest which are outside the control limit.

Claims

1. A computer-implemented method for data-driven anomaly detection, the method comprising:

receiving, by a processor, a data set comprising data captured from one or more sensors monitoring an automated system;

identifying, by the processor, a region of interest data subset of the input data set based on a dimensionality reduction technique and a change point detection algorithm, wherein the region of interest data subset has at least one different characteristic from remainder of the input data set;

mapping, by the processor, the region of interest data subset to a predefined group of reference data representing normal sensor data of a mode of operation of the automated system, wherein the mapping is based on closeness between a mean vector of the predefined group of the reference data and the data points within the region of interest data subset;

determining, by the processor, whether the data points within the region of interest data subset are outside of a predefined control limit of the corresponding mapped group; and

detecting, by the processor, at least one abnormal event by applying a heuristic algorithm on the data points within the region of interest which are outside the control limit.

2. The method as claimed in claim 1 further comprising:

determining an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

3. The method as claimed in claim 1, wherein the data is captured from the one or more sensors in real time.

4. The method as claimed in claim 1, wherein at least one of the one or more sensors is identified which corresponds to the data within the region of interest and falls outside of the control limit by applying the heuristic algorithm.

5. The method as claimed in claim 1, wherein the dimensionality reduction technique includes T² statistic.

6. The method as claimed in claim 1, wherein the one or more groups are classified based on one or more clustering algorithms.

7. The method as claimed in claim 1 further comprising:

generating an alert on the occurrence of the at least one abnormal event.

8. A system for data-driven anomaly detection, comprising:

a plurality of sensors monitoring an automated system;

a processor in operable communication with a processor-readable storage medium, the processor-readable storage medium containing one or more programming instructions whereby the processor is configured to implement;

a region of interest identification module configured to receive input data obtained from the plurality of sensors and identify a region of interest data subset from the input data based on a dimensionality reduction technique and a change point detection algorithm, wherein the region of interest data subset is a portion of the input data;

a mapping module configured to map the region of interest data subset with one or more predefined groups of reference data representing one or more modes of operation of a system based on closeness between a mean vector of the respective one or more predefined groups of reference data and the data within the region of interest data subset, wherein the reference data represent normal operating conditions of the system;

a data analysis module configured to determine whether the data points within the region of interest data subset are outside of a control limit calculated based on the corresponding mapped group of the one or more predefined groups, wherein the determination identifies if the data within the region of interest is below a lower control limit or above an upper control limit; and

an abnormal event detection module configured to detect at least one abnormal event by applying a heuristic algorithm on the data points within the region of interest data subset which are outside the control limit.

9. The system as claimed in claim 8 further comprising:

an earliest time detection module configured to determine an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

10. The system as claimed in claim 8, wherein the data captured from the plurality of sensors is preprocessed to remove incomplete data.

11. The system as claimed in claim 8, wherein at least one of the one or more sensors is identified which corresponds to the data within the region of interest and falls outside of the control limit by applying the heuristic algorithm.

12. The system as claimed in claim 8, wherein the dimensionality reduction technique includes T² statistic.

13. The system as claimed in claim 8, wherein, the one or more groups are classified based on one or more clustering algorithms.

14. The system as claimed in claim 8 further comprising:

an alert generation module configured to generate an alert on the occurrence of the at least one abnormal event.

15. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon for data-driven anomaly detection, the instructions comprising:

instructions for identifying a region of interest data subset from input data captured from one or more sensors monitoring an automated system based on a dimensionality reduction technique and a change point detection algorithm, wherein the region of interest data subset is a data subset of the input data and is calculated at least in part from a multi-modal pattern via cumulative sums of differences from the mean of the input data;

instructions for mapping the region of interest data subset with at least one group of reference data from one or more predefined groups of reference data representing one or more modes of operation of a system based on closeness between a mean vector of the respective groups of reference data and the data points within the region of interest data subset, wherein the reference data represent normal operating conditions of the system;

instructions for determining whether the data points within the region of interest data subset are outside of a control limit of the corresponding mapped at least one group of reference data of the one or more predefined groups of reference data, wherein the determination identifies if the data points within the region of interest data subset are below a lower control limit or above an upper control limit as determined based on the data of the corresponding mapped at least one group of reference data; and

instructions for detecting at least one abnormal event by applying a heuristic algorithm on the data points within the region of interest data subset which are outside the control limit.

16. The non-transitory computer-readable storage medium as claimed in claim 15 further comprising:

instructions for determining an earliest detection time of the at least one abnormal event by identifying a pattern of abnormality in the data within the region of interest which are outside the control limit.

17. The non-transitory computer-readable storage medium as claimed in claim 15 further comprising:

instructions for generating an alert on the occurrence of the at least one abnormal event.

18. The non-transitory computer-readable storage medium as claimed in claim 15 further comprising:

instructions for consolidating results and preparing a summary about the at least one abnormal event.

19. The non-transitory computer-readable storage medium as claimed in claim 15, wherein if no reference data is available, then the one or more predefined groups of reference data is obtained from a region outside the region of interest of the data itself.

20. The non-transitory computer-readable storage medium as claimed in claim 15, wherein the dimensionality reduction technique is applied first to the data and then the change point detection algorithm is applied to the dimensionally-reduced data.