SYSTEM AND METHOD FOR RANKING ANOMALIES

• US 20130110761A1
• Filed: 10/31/2011
• Published: 05/02/2013
• Est. Priority Date: 10/31/2011
• Status: Active Grant

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Krishnamurthy Viswanathan
• Inventors: Lakshminarayan, Choudur, Satterfield, Wade J., Talwar, Vanish, Wang, Chengwei, VISWANATHAN, Krishnamurthy
• Granted Patent: US 9,141,914 B2
• Time in Patent Office: Days
• US Class Current: 706/52
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Probable anomalies associated with at least one data metric may be detected across a series of windows of time series data by comparison of data to a threshold. An estimated probability of anomalies for each of the windows of time series data may be determined based on the detected probable anomalies and the threshold. The windows of time series data may be ranked based on the estimated probabilities. Probable anomalies associated with highest ranked windows of time series data may be output to a user.

Claims

1. A method comprising:

detecting, across a series of windows of time series data, probable anomalies associated with at least one data metric by comparison of data to a threshold;

determining an estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold; and

ranking the windows of time series data based on the estimated probabilities.

2. The method of claim 1, wherein determining an estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold comprises determining estimated probabilities using Gaussian approximation based on Z-scores, the Z-scores calculated using at least:

one sample mean; and

one sample standard deviation.

3. The method of claim 2, wherein the threshold comprises a threshold based on the mean and standard deviation of an underlying time series data distribution.

4. The method of claim 1, wherein:

detecting, across the series of windows of time series data, probable anomalies associated with the at least one data metric by comparison of data to the threshold comprises detecting a number data samples which exceed the threshold; and

determining an estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold comprises determining estimated probabilities using Bernoulli approximation based on the number of samples.

5. The method of claim 1, wherein ranking the windows of the time series data based the estimated probabilities comprises ranking the windows of the time series data from lowest estimated probability of anomaly to highest estimated probability of anomaly, wherein a window of time series data associated with the lowest estimated probability comprises a highest ranked window of time series data.

6. The method of claim 1, wherein:

determining estimated probabilities of anomalies for the windows of time series data based on the detected probable anomalies and the threshold comprises;

determining, using local anomaly detection systems, the estimated probabilities using Gaussian approximation based on Z-scores, at least one sample mean, and at least one sample standard deviation; and

outputting the estimated probabilities to a central node; and

ranking the windows of time series data based the estimated probabilities comprises ranking, by a central node, the estimate probabilities output from the local anomaly detection systems.

7. The method of claim 1, wherein:

detecting, across a series of windows of time series data, probable anomalies associated with at least one data metric by comparison of data to a threshold comprises;

determining, using local anomaly detection systems, a number data samples which exceed the threshold; and

outputting the number of data samples to a central node;

determining an estimated probability of anomalies comprises determining, by the central node, estimated probabilities using Bernoulli approximation based on the number of samples; and

ranking the windows of time series data based the estimated probabilities for the windows comprises ranking, by a central node, the estimated probabilities output from the local anomaly detection systems.

8. The method of claim 1, comprising outputting probable anomalies associated with highest ranked windows of time series data to a user.

9. The method of claim 1, wherein the series of windows of time series data comprises time series data associated with a data center.

10. The method of claim 1, wherein:

determining the estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold comprises determining the estimated probability of anomalies for each of the most recent windows of time series data for each of a plurality of monitored time series; and

ranking the windows of time series data based on the estimated probabilities comprises ranking the windows of the time series data based on the estimated probabilities for each of the most recent windows.

11. A system comprising:

a memory; and

a processor to;

detect, across a series of windows of time series data, probable anomalies associated with at least one data metric by comparison of data to a threshold;

determine an estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold; and

rank the windows of time series data based on the estimated probabilities.

12. The system of claim 11 wherein to determine an estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold the processor is to determine estimated probabilities using Gaussian approximation based on Z-scores, the Z-scores calculated using at least:

one sample mean; and

one sample standard deviation.

13. The system of claim 11, wherein:

to detect, across the series of windows of time series data, probable anomalies associated with the at least one data metric by comparison of data to the threshold the processor is to detect a number data samples which exceed the threshold; and

to determine an estimated probability of anomalies for each of the windows of time series data based on the detected probable anomalies and the threshold the processor is to determine estimated probabilities using Bernoulli approximation based on the number of samples.

14. The system of claim 11, wherein to rank the windows of the time series data based on the estimated probabilities the processor is to rank the windows of the time series data from lowest estimated probability of anomaly to highest estimated probability of anomaly, wherein a window of time series data associated with the lowest estimated probability comprises a highest ranked window of time series data.

15. The system of claim 11, wherein the threshold comprises a threshold based on a desired false positive rate.