Techniques for monitoring transformation techniques using control charts

• US 9,208,209 B1
• Filed: 03/25/2015
• Issued: 12/08/2015
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Katz, Joseph H.
• Primary Examiner(s): Le, Toan
• Application Number:
  US14/668,854
• Time in Patent Office: 258 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 5/025 Extracting rules from data

Abstract

Techniques for applying transformation techniques and using transformation outputs to implement remedial actions are included. A system may include a receiver that may receive time series data including data points and one or more specifications for a transformation technique applicable to the time series data. The system may include a transformation engine that may apply the transformation technique to the time series data to produce outputs. The system may include a control engine that may determine a set of control limits using, for example, a control chart and a set of residual values. The system may include an anomaly detection engine that may generate the set of residual values using the current data points and the outputs. The anomaly detection engine may further identify one or more anomalies of the set of residual values based on the set of control limits and the control chart.

Claims

1. A forecasting system for applying forecasts and using outputs to implement remedial actions, comprising:

a receiver that receives;

time series data including a plurality of current data points; and

a forecast model for a forecast applicable to the time series data;

a transformation engine that applies the forecast model to the time series data to produce a plurality of outputs;

a control engine that determines a set of control limits using a control chart and a set of residual values;

an anomaly detection engine that;

generates the set of residual values using the plurality of current data points and the plurality of outputs, wherein each residual value in the set of residual values indicates a difference between a current data point of the plurality of current data points and an output of the plurality of outputs; and

identifies one or more anomalies of the set of residual values based on the set of control limits and the control chart; and

a remedial-action engine that;

determines a remedial action list including a plurality of types of remedial actions;

assigns a prioritization to the types of remedial actions in the remedial action list, wherein prioritizations are based on computational costs of the remedial actions;

determines a remedial action for the forecast model based on the prioritizations of the types of remedial actions in the remedial action list and the one or more anomalies; and

perform the remedial action related to the forecast model.

2. The system of claim 1, wherein an anomaly indicates that a residual value of the set of residual values occurs outside the set of control limits.

3. The system of claim 1, wherein an anomaly indicates that a subset of sequentially residual values incrementally increase.

4. The system of claim 1, wherein identifying one or more anomalies includes determining a vintage date,

wherein a vintage date indicates a date at which the forecast model was selected or modified, and

wherein the one or more anomalies are identified for dates occurring on or after the vintage date.

5. The system of claim 1, wherein identifying one or more anomalies includes analyzing the set of residual values using a set of rules, and

wherein analyzing the set of residual values using the set of rules includes determining a non-random condition in a control chart.

6. The system of claim 1, wherein determining a remedial action for the forecast model further comprises:

determining a classification for each of the one or more anomalies,

wherein a classification is associated with a remedial action;

sorting the classifications using a predetermined prioritization scheme; and

retaining a classification of the sorted classifications,

wherein the determined remedial action is associated with the retained classification.

7. The system of claim 1, wherein remedial actions include estimating parameters for the forecast model, applying the forecast model to the time series data to produce a plurality of outputs, or selecting a different forecast model.

8. The system of claim 7, wherein identifying one or more anomalies includes determining that the tracking signal equation has produced an output that exceeds one or more error threshold values.

9. The system of claim 1, wherein identifying one or more anomalies includes analyzing the set of residual values using a tracking signal equation,

wherein the tracking signal equation uses a sum of the set of residual values and a mean absolute deviation value, and

wherein analyzing the set of residual values using the tracking signal equation includes determining a non-random condition in a control chart.

10. The system of claim 8, wherein identifying one or more anomalies includes analyzing the set of residual values using the tracking signal equation further includes converting a product of the tracking signal equation using a standard deviation value.

11. A non-transitory computer program product, tangible embodied in a non-transitory machine readable storage medium, including instructions operable to cause a data processing apparatus to:

receive time series data including a plurality of current data points;

receive a forecast model for a forecast applicable to the time series data;

apply the forecast model to the time series data to produce a plurality of outputs;

generate a set of residual values using the plurality of current data points and the plurality of outputs, wherein a residual value indicates a difference between a current data point of the plurality of data points and an output of the plurality of outputs;

determine a set of control limits using a control chart and the set of residual values;

identify one or more anomalies of the set of residual values, wherein anomalies are identified using the set of control limits and the control chart;

determine a remedial action list including a plurality of types of remedial actions;

assign a prioritization to the types of remedial actions in the remedial action list, wherein prioritizations are based on computational costs of the remedial actions;

determine a remedial action for the forecast model based on the prioritizations of the types of remedial actions in the remedial action list and the one or more anomalies; and

perform the remedial action related to the forecast model.

12. The computer program product of claim 11, wherein an anomaly indicates that a residual value of the set of residual values occurs outside the set of control limits.

13. The computer program product of claim 11, wherein an anomaly indicates that a subset of sequentially residual values incrementally increase.

14. The computer program product of claim 11, wherein identifying one or more anomalies includes determining a vintage date,

wherein a vintage date indicates a date at which the forecast model was selected or modified, and

wherein the one or more anomalies are identified for dates occurring on or after the vintage date.

15. The computer program product of claim 11, wherein identifying one or more anomalies includes analyzing the set of residual values using a set of rules, and

wherein analyzing the set of residual values using the set of rules includes determining a non-random condition in a control chart.

16. The computer program product of claim 11, wherein instructions for determining a remedial action includes further instructions operable to cause a data processing apparatus to:

determine a classification for each of the one or more anomalies,

wherein a classification is associated with a remedial action;

sort the classifications using a predetermined prioritization scheme; and

retain a classification of the sorted classifications,

wherein the determined remedial action is associated with the retained classification.

17. The computer program product of claim 11, wherein remedial actions estimating parameters for the forecast model, applying the forecast model to the time series data to produce a plurality of outputs, or selecting a different forecast model.

18. The computer program product of claim 11, wherein identifying one or more anomalies includes analyzing the set of residual values using a tracking signal equation,

wherein the tracking signal equation uses a sum of the set of residual values and a mean absolute deviation value, and

wherein analyzing the set of residual values using the tracking signal equation includes determining a non-random condition in a control chart.

19. The computer program product of claim 18, wherein identifying one or more anomalies includes analyzing the set of residual values using the tracking signal equation further includes converting a product of the tracking signal equation using a standard deviation value.

20. The computer program product of claim 18, wherein identifying one or more anomalies includes determining that the tracking signal equation has produced an output that exceeds one or more error threshold values.

21. A computer-implemented method for applying forecast models and using outputs to implement remedial actions, comprising:

receiving time series data including a plurality of current data points;

receiving a forecast model for a forecast applicable to the time series data;

applying the forecast model to the time series data to produce a plurality of outputs;

generating, using an anomaly detection engine, a set of residual values using the plurality of current data points and the plurality of outputs, wherein a residual value indicates a difference between a current data point of the plurality of data points and an output of the plurality of outputs;

determining a set of control limits using a control chart and the set of residual values;

identifying, using the anomaly detection engine, one or more anomalies of the set of residual values, wherein anomalies are identified using the set of control limits and the control chart;

determining, using a remedial-action engine, a remedial action list including a plurality of types of remedial actions;

assigning, using the remedial-action engine, a prioritization to the types of remedial actions in the remedial action list, wherein prioritizations are based on computational costs of the remedial actions;

determining, using the remedial-action engine, a remedial action for the forecast model based on the prioritizations of the types of remedial actions in the remedial action list and the one or more anomalies; and

performing, using the remedial-action engine, the remedial action related to the forecast model.

22. The computer-implemented method of claim 21, wherein an anomaly indicates that a residual value of the set of residual values occurs outside the set of control limits.

23. The computer-implemented method of claim 21, wherein an anomaly indicates that a subset of sequentially residual values incrementally increase.

24. The computer-implemented method of claim 21, wherein identifying one or more anomalies includes determining a vintage date,

wherein a vintage date indicates a date at which the forecast model was selected or modified, and

wherein the one or more anomalies are identified for dates occurring on or after the vintage date.

25. The computer-implemented method of claim 21, wherein identifying one or more anomalies includes analyzing the set of residual values using a set of rules, and

wherein analyzing the set of residual values using the set of rules includes determining a non-random condition in a control chart.

26. The computer-implemented method of claim 21, wherein determining a remedial action further comprises:

determining a classification for each of the one or more anomalies,

wherein a classification is associated with a remedial action;

sorting the classifications using a predetermined prioritization scheme; and

retaining a classification of the sorted classifications,

wherein the determined remedial action is associated with the retained classification.

27. The computer-implemented method of claim 21, wherein remedial actions include estimating parameters for the forecast model, applying the forecast model to the time series data to produce a plurality of outputs, or selecting a different forecast model.

28. The computer-implemented method of claim 21, wherein identifying one or more anomalies includes analyzing the set of residual values using a tracking signal equation,

wherein the tracking signal equation uses a sum of the set of residual values and a mean absolute deviation value, and

wherein analyzing the set of residual values using the tracking signal equation includes determining a non-random condition in a control chart.

29. The computer-implemented method of claim 28, wherein identifying one or more anomalies includes analyzing the set of residual values using the tracking signal equation further includes converting a product of the tracking signal equation using a standard deviation value.

30. The computer-implemented method of claim 28, wherein identifying one or more anomalies includes determining that the tracking signal equation has produced an output that exceeds one or more error threshold values.