Configuration of continuous anomaly detection service

• US 10,146,609 B1
• Filed: 07/08/2016
• Issued: 12/04/2018
• Est. Priority Date: 07/08/2016
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Leverich, Jacob Barton, Cai, Shang, Zhang, Hongyang, Ganea, Mihai, Cruise, Alex
• Primary Examiner(s): Bryan, Jason B
• Application Number:
  US15/206,126
• Time in Patent Office: 879 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40

Abstract

A continuous anomaly detection service receives data stream and performs continuous anomaly detection on the incoming data streams. This continuous anomaly detection is performed based on anomaly detection definitions, which define a signal used for anomaly detection and an anomaly detection configuration. These anomaly detection definitions can be modified, such that continuous anomaly detection continues to be performed for the data stream and the signal, based on the new anomaly detection definition.

Claims

1. A computer-implemented method for configuring a continuous anomaly detection service, comprising:

accessing a plurality of anomaly detection definitions, wherein each of the plurality of anomaly detection definitions comprises an anomaly detection configuration that defines how to generate and store, for a corresponding signal, a sequential set of data points from a plurality of historical data points not stored in the sequential set of data points;

receiving a request to modify a first anomaly detection definition of the plurality of anomaly detection definitions, wherein the first anomaly detection definition comprises a first anomaly detection configuration for a corresponding first signal;

updating the first anomaly detection configuration based on the request to modify;

updating the first signal, based on the updated first anomaly detection configuration, by replacing data points of the sequential set of data points of the first signal with a subset of the plurality of historical data points; and

continuously analyzing memory storing the signals to determine a corresponding anomaly result for each of the plurality of anomaly detection definitions, wherein the anomaly result for the first anomaly detection definition is based on the updated first anomaly detection configuration.

2. The computer-implemented method of claim 1, wherein receiving a request to modify a first anomaly detection definition of the plurality of anomaly detection definitions comprises:

receiving the request to modify the first anomaly detection definition; and

determining, from the request to modify, which of the plurality of anomaly detection definitions comprises the first anomaly detection definition.

3. The computer-implemented method of claim 2, wherein the first anomaly detection configuration comprises a plurality of first configuration parameters, wherein the request to modify includes revised values for one or more of the first configuration parameters, and wherein updating the first anomaly detection configuration based on the request to modify comprises updating the one or more of the first configuration parameters with the revised values.

4. The computer-implemented method of claim 1, wherein a first configuration object of a plurality of configuration objects includes all of the plurality of anomaly detection definitions.

5. The computer-implemented method of claim 4, further comprising a second configuration object and second plurality of anomaly detection definitions, wherein the second configuration object includes all of the second plurality of anomaly detection definitions.

6. The computer-implemented method of claim 4, wherein the first configuration object defines a shared search for the plurality of anomaly detection definitions.

7. The computer-implemented method of claim 1, further comprising:

determining, based on the updated first anomaly detection configuration, that the first signal requires modification; and

modifying the first signal based on the updated anomaly detection configuration.

8. The computer-implemented method of claim 7, wherein modifying the first signal comprises:

deleting all existing data points of the sequential set of data points of the first signal;

accessing the subset of the plurality of historical data points; and

replacing the deleted data points of the sequential set of data points with the subset of the plurality of historical data points.

9. The computer-implemented method of claim 7, wherein modifying the first signal comprises:

identifying target data points of the sequential set of data points of the first signal, wherein the target data points do not conform to the requirements of the updated first anomaly detection configuration;

identifying the subset of the plurality of historical data points to replace the target data points in the sequential set of data points; and

replacing the target data points with the subset of the plurality of historical data points.

10. The computer-implemented method of claim 9, wherein the replacement of the target data points with the subset of the historical data points results in a change in the sequence of the sequential set of data points.

11. The computer-implemented method of claim 1, wherein the anomaly detection configuration defines how to update the sequential set of data points with substantially real-time data.

12. The computer-implemented method of claim 11, wherein deleting the requested anomaly detection definition comprises deleting the sequential set of data points of the signal of the deleted anomaly detection definition.

13. The computer-implemented method of claim 1, further comprising:

receiving a request to create a new anomaly detection definition, wherein the request includes a new anomaly detection configuration for a new signal of the new anomaly detection definition;

determining a corresponding subset of the plurality of historical data points that are required for a new sequential set of data points of the new signal based on the new anomaly detection configuration;

accessing the corresponding subset of the historical data points; and

generating the new signal based on the corresponding subset of the historical data points and the new anomaly detection configuration.

14. The computer-implemented method of claim 13, further comprising determining an anomaly result for the new anomaly detection definition based on the new signal and the new anomaly detection configuration.

15. The computer-implemented method of claim 1, further comprising:

receiving a plurality of data streams;

determining, for each of the anomaly detection definitions, that one of the plurality of data streams is associated with the corresponding signal for the anomaly detection definition;

identifying, for each of the anomaly detection definitions, one or more data points of the associated data stream; and

updating, for each of the anomaly detection definitions, the sequential set of data points of the corresponding signal with the one or more data points.

16. The computer-implemented method of claim 15, wherein determining, for each of the anomaly detection definitions, that one of the plurality of data streams that is associated with the corresponding signal for the anomaly detection definition comprises performing a search of the plurality of data streams based on a plurality of key performance indicators, wherein each signal is associated with one or more of the key performance indicators.

17. The computer-implemented method of claim 1, wherein each anomaly detection configuration comprises a plurality of configuration parameters, and wherein the configuration parameters comprise an anomaly detection threshold, a signal length, a signal resolution, an associated key performance indicator, an alert setting, or an anomaly detection procedure.

18. The computer-implemented method of claim 1, wherein each anomaly detection configuration comprises an anomaly detection procedure, and wherein the anomaly detection procedure comprises a trending anomaly detection procedure or a cohesive anomaly detection procedure.

19. A non-transitory computer-readable storage medium comprising instructions stored thereon, which when executed by one or more processors, cause the one or more processors to perform operations comprising:

accessing a plurality of anomaly detection definitions, wherein each of the plurality of anomaly detection definitions comprises an anomaly detection configuration that defines how to generate and store, for a corresponding signal, a sequential set of data points from a plurality of historical data points not stored in the sequential set of data points;

receiving a request to modify a first anomaly detection definition of the plurality of anomaly detection definitions, wherein the first anomaly detection definition comprises a first anomaly detection configuration for a corresponding first signal;

updating the first anomaly detection configuration based on the request to modify;

updating the first signal, based on the updated first anomaly detection configuration, by replacing data points of the sequential set of data points of the first signal with a subset of the plurality of historical data points; and

continuously analyzing memory storing the signals to determine corresponding anomaly result for each of the plurality of anomaly detection definitions, wherein the anomaly result for the first anomaly detection definition is based on the updated first anomaly detection configuration.

20. A system for performing continuous anomaly detection, comprising:

at least one memory having instructions stored thereon; and

at least one processor configured to execute the instructions to;

access a plurality of anomaly detection definitions, wherein each of the plurality of anomaly detection definitions comprises an anomaly detection configuration that defines how to generate and store, for a corresponding signal, a sequential set of data points from a plurality of historical data points not stored in the sequential set of data points;

receive a request to modify a first anomaly detection definition of the plurality of anomaly detection definitions, wherein the first anomaly detection definition comprises a first anomaly detection configuration for a corresponding first signal;

update the first anomaly detection configuration based on the request to modify;

update the first signal, based on the updated first anomaly detection configuration, by replacing data points of the sequential set of data points of the first signal with a subset of the plurality of historical data points; and

continuously analyze a portion of the at least one memory storing the signals to determine a corresponding anomaly result for each of the plurality of anomaly detection definitions, wherein the anomaly result for the first anomaly detection definition is based on the updated first anomaly detection configuration.