Anomaly detection in performance management

• US 10,223,191 B2
• Filed: 07/20/2016
• Issued: 03/05/2019
• Est. Priority Date: 07/20/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Della Corte, Gianluca, Gianfagna, Leonida, Proietti, Stefano, Pecoraro, Roberto, Sgro, Antonio M.
• Primary Examiner(s): Manoskey, Joseph D
• Application Number:
  US15/214,917
• Publication Number:
  US 20180024875A1
• Time in Patent Office: 958 Days
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis

Abstract

Methods and systems for detecting anomalous behavior include performing a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes. The KPIs are clustered in a space defined by the set of principal axes. Local and structural anomalies are determined in the clustered KPIs. The structural and local anomalies are classified based on historical information. A management action is performed based on the classified structural and local anomalies.

Claims

1. A method for detecting anomalous behavior, comprising:

performing a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes;

clustering the KPIs in a space defined by the set of principal axes;

determining structural anomalies in the clustered KPIs based on whether a cluster of KPIs that does not conform to a Gaussian distribution had conformed to a Gaussian distribution at a previous time;

determining local anomalies in the clustered KPIs;

classifying the structural and local anomalies based on historical information; and

performing a management action based on the classified structural and local anomalies.

2. The method of claim 1, wherein determining structural anomalies comprises determining how closely each cluster of KPIs conforms to a Gaussian distribution.

3. The method of claim 1, wherein determining local anomalies comprises comparing a distance of each individual KPI from its respective cluster mean to a threshold.

4. The method of claim 3, wherein determining local anomalies comprises performing said comparison only for KPIs in clusters that conform to a Gaussian distribution.

5. The method of claim 3, wherein classifying local anomalies comprises determining whether a KPI is an isolated outlier.

6. The method of claim 5, wherein classifying local anomalies comprises determining whether the KPI has gone from a distance below a first threshold at a previous sampling time to a current distance that is above a second, higher threshold.

7. The method of claim 5, wherein classifying local anomalies comprises determining whether there is a trend across multiple previous sampling times from a distance below a first threshold toward a second, higher threshold.

8. The method of claim 1, further comprising transforming from a space based on the principal axes to an original space and determining whether each of the local and structural anomalies is a global anomaly or a local anomaly.

9. A non-transitory computer readable storage medium comprising a computer readable program for detecting anomalous behavior, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:

performing a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes;

clustering the KPIs in a space defined by the set of principal axes;

determining structural anomalies in the clustered KPIs based on whether a cluster of KPIs that does not conform to a Gaussian distribution had conformed to a Gaussian distribution at a previous time;

determining local anomalies in the clustered KPIs;

classifying the structural and local anomalies based on historical information; and

performing a management action based on the classified structural and local anomalies.

10. A system for detecting anomalous behavior, comprising:

a detection module comprising a processor configured to perform a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes, to cluster the KPIs in a space defined by the set of principal axes, to determine structural anomalies in the clustered KPIs based on whether a cluster of KPIs that does not conform to a Gaussian distribution had conformed to a Gaussian distribution at a previous time, and to determine local anomalies in the clustered KPIs;

a classification module configured to classify the structural and local anomalies based on historical information; and

a management module configured to perform a management action based on the classified structural and local anomalies.

11. The system of claim 10, wherein the detection module is further configured to determine how closely each cluster of KPIs conforms to a Gaussian distribution.

12. The system of claim 10, wherein the detection module is further configured to compare a distance of each individual KPI from its respective cluster mean to a threshold.

13. The system of claim 12, wherein the classification module is further configured to perform said comparison only for KPIs in clusters that conform to a Gaussian distribution.

14. The system of claim 12, wherein the classification module is further configured to determine whether a KPI is an isolated outlier.

15. The system of claim 14, wherein the classification module is further configured to determine whether the KPI has gone from a distance below a first threshold at a previous sampling time to a current distance that is above a second, higher threshold.

16. The system of claim 14, wherein the classification module is further configured to determine whether there is a trend across multiple previous sampling times from a distance below a first threshold toward a second, higher threshold.

17. The system of claim 10, wherein the classification module is further configured to transform from a space based on the principal axes to an original space and to determine whether each of the local and structural anomalies is a global anomaly or a local anomaly.