ANOMALY DETECTION FOR NON-STATIONARY DATA

• US 20160189041A1
• Filed: 12/31/2014
• Published: 06/30/2016
• Est. Priority Date: 12/31/2014
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bawa, Gagandeep Singh, Schwarzbach, David, Moghtaderi, Azadeh
• Granted Patent: US 10,445,644 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

A method of detecting anomalies in a time series is disclosed. A training time series corresponding to a process is extracted from an initial time series corresponding to the process, the training time series including a subset of the initial time series. Outlier data points in the training time series are modified based on predetermined acceptability criteria. A plurality of prediction methods are trained using the training time series. An actual data point corresponding to the initial time series is received. The plurality of prediction methods are used to determine a set of predicted data points corresponding to the actual data point. It is determined whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point.

Claims

1. A method comprising:

extracting a training time series corresponding to a process from an initial time series corresponding to the process, the training time series including a subset of the initial time series, the subset of the initial time series having a length offset by an index prior to a last data point of the initial time series;

modifying outlier data points in the training time series based on predetermined acceptability criteria;

training a plurality of prediction methods using the training time series;

receiving an actual data point corresponding to the initial time series, the actual data point having an index after the last data point of the training time series;

using the plurality of prediction methods to determine a set of predicted data points corresponding to the actual data point of the initial time series; and

determining whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point.

2. The method of claim 1, wherein the calculation of whether each of the set of predicted data points is statistically different from the actual data point includes a determination that the Mahalanobis distance between the prediction error and the fitted multivariate normal joint probability distribution of each of the set of predicted data points is within a specified range.

3. The method of claim 1, further comprising receiving an additional actual data point corresponding to the initial time series and extracting an additional training time series having the length offset by an additional index prior to a last data point of the initial time series the additional index reflecting a relative position of the actual data point to the additional actual data point.

4. The method of claim 1, further comprising selecting the combination of each of the plurality of prediction methods to minimize a number of false anomaly detections.

5. The method of claim 1, further comprising representing the determination of whether the actual data point is anomalous in a graphical user interface, the representing including providing a strength of the determination.

6. The method of claim 5, wherein the strength of the determination is based on a number of the plurality of prediction methods that indicate an anomaly with respect to the data point.

7. The method of claim 1, wherein the training time series represents a window of the initial time series that is recent in relation to the actual data point.

8. A system comprising:

one or more modules implemented by one or more processors, the one or more modules configured to;

extract a training time series corresponding to a process from an initial time series corresponding to the process, the training time series including a subset of the initial time series, the subset of the initial time series having a length offset by an index prior to a last data point of the initial time series;

modify outlier data points in the training time series based on predetermined acceptability criteria;

train a plurality of prediction methods using the training time series;

receive an actual data point corresponding to the initial time series, the actual data point having an index after the last data point of the training time series;

use the plurality of prediction methods to determine a set of predicted data points corresponding to the actual data point of the initial time series; and

determine whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point.

9. The system of claim 8, wherein the calculation of whether each of the set of predicted data points is statistically different from the actual data point includes a determination that the Mahalanobis distance between the prediction error and the fitted multivariate normal joint probability distribution of each of the set of predicted data points is within a specified range.

10. The system of claim 8, further comprising receiving an additional actual data point corresponding to the initial time series and extracting an additional training time series having the length offset by an additional index prior to a last data point of the initial time series the additional index reflecting a relative position of the actual data point to the additional actual data point.

11. The system of claim 8, further comprising selecting the combination of each of the plurality of prediction methods to minimize a number of false anomaly detections.

12. The system of claim 8, further comprising representing the determination of whether the actual data point is anomalous in a graphical user interface, the representing including providing a strength of the determination.

13. The system of claim 12, wherein the strength of the determination is based on a number of the plurality of prediction methods that indicate an anomaly with respect to the data point.

14. The system of claim 8, wherein the training time series represents a window of the initial time series that is recent in relation to the actual data point.

15. A non-transitory machine readable medium comprising a set of instructions that, when executed by a processor, causes the processor to perform operations, the operations comprising:

extracting a training time series corresponding to a process from an initial time series corresponding to the process, the training time series including a subset of the initial time series, the subset of the initial time series having a length offset by an index prior to a last data point of the initial time series;

modifying outlier data points in the training time series based on predetermined acceptability criteria;

training a plurality of prediction methods using the training time series;

receiving an actual data point corresponding to the initial time series, the actual data point having an index after the last data point of the training time series;

using the plurality of prediction methods to determine a set of predicted data points corresponding to the actual data point of the initial time series; and

determining whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point.

16. The non-transitory machine readable medium of claim 15, wherein the calculation of whether each of the set of predicted data points is statistically different from the actual data point includes a determination that the Mahalanobis distance between the prediction error and the fitted multivariate normal joint probability distribution of each of the set of predicted data points is within a specified range.

17. The non-transitory machine readable medium of claim 15, further comprising receiving an additional actual data point corresponding to the initial time series and extracting an additional training time series having the length offset by an additional index prior to a last data point of the initial time series the additional index reflecting a relative position of the actual data point to the additional actual data point.

18. The non-transitory machine readable medium of claim 15, further comprising selecting the combination of each of the plurality of prediction methods to minimize a number of false anomaly detections.

19. The non-transitory machine readable medium of claim 15, further comprising representing the determination of whether the actual data point is anomalous in a graphical user interface, the representing including providing a strength of the determination.

20. The non-transitory machine readable medium of claim 19, wherein the strength of the determination is based on a number of the plurality of prediction methods that indicate an anomaly with respect to the data point.