ANOMALY DETECTION FOR NON-STATIONARY DATA

• US 20170372207A1
• Filed: 06/30/2017
• Published: 12/28/2017
• Est. Priority Date: 12/31/2014
• Status: Abandoned Application

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Moghtaderi, Azadeh, Bawa, Gagandeep Singh, Schwarzbach, David
• Application Number:
  US15/640,191
• Publication Number:
  US 20170372207A1
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

A method of detecting anomalies in a time series is disclosed. A training time series corresponding to a process is extracted from an initial time series corresponding to the process, the training time series including a subset of the initial time series. Outlier data points in the training time series are modified based on predetermined acceptability criteria. A plurality of prediction methods are trained using the training time series. An actual data point corresponding to the initial time series is received. The plurality of prediction methods are used to determine a set of predicted data points corresponding to the actual data point. It is determined whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point.

Claims

1. A system comprising:

one or more computer processors;

one or more computer memories;

one or more modules incorporated into the one or more computer memories, the one or more modules configuring the one or more computer processors to perform operations, the operations comprising;

extracting a training time series corresponding to a process from an initial time series corresponding to the process;

modifying outlier data points in the training time series based on predetermined acceptability criteria;

training a plurality of prediction methods using the training time series;

receiving an actual data point corresponding to the initial time series;

using the plurality of prediction methods to determine a set of predicted data points corresponding to the actual data point of the initial time series;

determining whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point; and

receiving an additional actual data point corresponding to the initial time series and extracting an additional training time series from the initial time series based on the additional actual data point.

2. The system of claim 1, wherein the calculation of whether each of the set of predicted data points is statistically different from the actual data point includes a determination that the Mahalanobis distance between the prediction error and the fitted multivariate normal joint probability distribution of each of the set of predicted data points is within a specified range.

3. The system of claim 1, wherein the additional actual data point corresponds to the initial time series and the operations further comprise extracting an additional training time series having the length offset by an additional index prior to a last data point of the initial time series the additional index reflecting a relative position of the actual data point to the additional actual data point.

4. The system of claim 1, further comprising selecting the combination of each of the plurality of prediction methods to minimize a number of false anomaly detections.

5. The system of claim 1, further comprising representing the determination of whether the actual data point is anomalous in a graphical user interface, the representing including providing a strength of the determination.

6. The system of claim 5, wherein the strength of the determination is based on a number of the plurality of prediction methods that indicate an anomaly with respect to the data point.

7. The system of claim 1, wherein the training time series represents a window of the initial time series that is recent in relation to the actual data point.

8. A method comprising:

extracting a training time series corresponding to a process from an initial time series corresponding to the process;

modifying outlier data points in the training time series based on predetermined acceptability criteria;

training a plurality of prediction methods using the training time series;

receiving an actual data point corresponding to the initial time series;

using the plurality of prediction methods to determine a set of predicted data points corresponding to the actual data point of the initial time series;

determining whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point; and

receiving an additional actual data point corresponding to the initial time series and extracting an additional training time series from the initial time series based on the additional actual data point.

9. The method of claim 8, wherein the calculation of whether each of the set of predicted data points is statistically different from the actual data point includes a determination that the Mahalanobis distance between the prediction error and the fitted multivariate normal joint probability distribution of each of the set of predicted data points is within a specified range.

10. The method of claim 8, wherein additional actual data point corresponds to the initial time series and the method further comprises extracting an additional training time series having the length offset by an additional index prior to a last data point of the initial time series the additional index reflecting a relative position of the actual data point to the additional actual data point.

11. The method of claim 8, further comprising selecting the combination of each of the plurality of prediction methods to minimize a number of false anomaly detections.

12. The method of claim 8, further comprising representing the determination of whether the actual data point is anomalous in a graphical user interface, the representing including providing a strength of the determination.

13. The method of claim 12, wherein the strength of the determination is based on a number of the plurality of prediction methods that indicate an anomaly with respect to the data point.

14. The method of claim 8, wherein the training time series represents a window of the initial time series that is recent in relation to the actual data point.

15. A non-transitory machine readable medium comprising a set of instructions that, when executed by a processor, causes the processor to perform operations, the operations comprising:

extracting a training time series corresponding to a process from an initial time series corresponding to the process;

modifying outlier data points in the training time series based on predetermined acceptability criteria;

training a plurality of prediction methods using the training time series;

receiving an actual data point corresponding to the initial time series;

using the plurality of prediction methods to determine a set of predicted data points corresponding to the actual data point of the initial time series;

determining whether the actual data point is anomalous based on a calculation of whether each of the set of predicted data points is statistically different from the actual data point; and

receiving an additional actual data point corresponding to the initial time series and extracting an additional training time series from the initial time series based on the additional actual data point.

16. The non-transitory machine readable medium of claim 15, wherein the calculation of whether each of the set of predicted data points is statistically different from the actual data point includes a determination that the Mahalanobis distance between the prediction error and the fitted multivariate normal joint probability distribution of each of the set of predicted data points is within a specified range.

17. The non-transitory machine readable medium of claim 15, wherein the additional actual data point corresponds to the initial time series and the operations further comprise extracting an additional training time series having the length offset by an additional index prior to a last data point of the initial time series the additional index reflecting a relative position of the actual data point to the additional actual data point.

18. The non-transitory machine readable medium of claim 15, the operations further comprising selecting the combination of each of the plurality of prediction methods to minimize a number of false anomaly detections.

19. The non-transitory machine readable medium of claim 15, the operations further comprising representing the determination of whether the actual data point is anomalous in a graphical user interface, the representing including providing a strength of the determination.

20. The non-transitory machine readable medium of claim 19, wherein the strength of the determination is based on a number of the plurality of prediction methods that indicate an anomaly with respect to the data point.