Adaptive key performance indicator thresholds

• US 10,235,638 B2
• Filed: 09/18/2015
• Issued: 03/19/2019
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Maheshwari, Sonal, Sainani, Manish, Alekseyev, Leonid, Hardin, Alan, Leverich, Jacob Barton, Oliner, Adam Jamison, Reyes, Brian, Bhide, Alok Anant
• Primary Examiner(s): Mirza, Adnan M
• Application Number:
  US14/859,236
• Publication Number:
  US 20160104076A1
• Time in Patent Office: 1,278 Days
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Techniques are disclosed for providing adaptive thresholding technology for Key Performance Indicators (KPIs). Adaptive thresholding technology may automatically assign new values or adjust existing values for one or more thresholds of one or more time policies. Assigning threshold values using adaptive thresholding may involve identifying training data (e.g., historical data, simulated data, or example data) for the time frames and analyzing the training data to identify variations within the data (e.g., patterns, distributions, trends). A threshold value may be determined based on the variations and may be assigned to one or more of the thresholds without additional user intervention.

Claims

1. A method comprising:

accessing information that defines one or more time frames associated with a key performance indicator (KPI), each of the time frames having a set of one or more thresholds, wherein each threshold represents the end of a range of values corresponding to a particular state of the KPI, and wherein the KPI is defined by a search query that derives a value indicative of the performance of a service at a point in time or during a period of time, the value derived from machine data pertaining to one or more entities that provide the service;

assigning one or more threshold values to the thresholds, the assigning the one or more threshold values comprising, for each time frame;

identifying training data for the time frame, and

determining the one or more threshold values for the time frame in consideration of the identified training data; and

wherein the assigning the one or more threshold values comprises assigning a first threshold value to a threshold and subsequently assigning a second threshold value to the threshold, wherein the first threshold value and the second threshold value are based on training data from different time durations;

wherein the method is performed by a computer system comprising one or more processors.

2. The method of claim 1, wherein assigning the one or more threshold values comprises assigning the one or more threshold values to the thresholds automatically based on a schedule, a frequency interval, or an event.

3. The method of claim 1, wherein determining the one or more threshold values further comprise:

analyzing the KPI values to determine a statistical metric indicating changes in the training data; and

updating the set of one or more thresholds for the one or more time frames.

4. The method of claim 1, wherein the training data comprises simulated data, historical data, or example data.

5. The method of claim 1, wherein the training data comprises simulated values, historical values, or example values of the KPI.

6. The method of claim 1, wherein the training data comprises training data that was generated by or about the one or more entities during a fixed duration of time.

7. The method of claim 1, wherein the training data is the most current historical data.

8. The method of claim 1, wherein the one or more time frames occur multiple times within a time cycle, wherein the time cycle is based on a daily time cycle, a weekly time cycle, or a monthly time cycle.

9. The method of claim 1, wherein determining one or more thresholds comprises determining a change to an existing threshold value, wherein the change is based on a delta value, a percentage value, or an absolute value.

10. The method of claim 1, further comprising causing for display a graphical user interface including a presentation schedule with one or more time slots corresponding to each of the time frames, the one or more time slots having a threshold marker for each of the one or more thresholds of the set.

11. The method of claim 1, further comprising causing for display a graphical user interface including a presentation schedule with a plurality of time slots, wherein one or more of the time slots correspond to a first time frame and have a unifying appearance to distinguish the one or more time slots from time slots corresponding to another time frame.

12. The method of claim 1, further comprising executing the search query defining the KPI to derive a KPI value and assigning the particular state of the KPI when the KPI value is within a range hounded by the one or more thresholds.

13. The method of claim 1, wherein the machine data is stored as time-stamped events.

14. The method of claim 1, wherein the machine data is stored as time-stamped events, where each time-stamped event includes a portion of raw machine data.

15. The method of claim 1, wherein the machine data is stored as time-stamped events including portions of raw machine data and is accessed using a late-binding schema.

16. The method of claim 1, wherein the search query uses a late-binding schema to extract values indicative of the performance of the service from time-stamped events after the search query is initiated.

17. The method of claim 1, wherein the machine data pertaining to the entity comprises heterogeneous machine data from multiple sources.

18. The method of claim 1, wherein the machine data pertaining to the entity comprises machine data from the entity and another entity.

19. A system comprising:

a memory; and

a processing device coupled with the memory to;

access information that defines one or more time frames associated with a key performance indicator (KPI), each of the time frames having a set of one or more thresholds wherein each threshold represents the end of a range of values corresponding to a particular state of the KPI, and wherein the KPI is defined by a search query that derives a value indicative of the performance of a service at a point in time or during a period of time, the value derived from machine data pertaining to one or more entities that provide the service;

assign one or more threshold values to the thresholds, comprising, for each time frame;

identify training data for the time frame, and

determine the one or more threshold values for the time frame in consideration of the identified training data; and

wherein the assigning the one or more threshold values comprises assigning a first threshold value to a threshold and subsequently assigning a second threshold value to the threshold, wherein the first threshold value and the second threshold value are based on training data from different time durations;

wherein the method is performed by a computer system comprising one or more processors.

20. The system of claim 19, wherein assigning the one or more threshold values comprises assigning the one or more threshold values to the thresholds automatically based on a schedule, a frequency interval, or an event.

21. The system of claim 19, wherein determining the one or more threshold values further comprise:

analyzing the KPI values to determine a statistical metric indicating changes in the training data; and

updating the set of one or more thresholds for the one or more time frames.

22. The system of claim 19, wherein the training data comprises simulated data, historical data, or example data.

28. The system of claim 19, wherein the training data comprises simulated values, historical values, or example values of the KPI.

29. The system of claim 19, wherein the training data comprises training data that was generated by or about the one or more entities during a fixed duration of time.

23. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by one or more processing devices, causes the processing device to perform operations comprising:

accessing information that defines one or more time frames associated with a key performance indicator (KPI), each of the time frames having a set of one or more thresholds wherein each threshold represents the end of a range of values corresponding to a particular state of the KPI, and wherein the KPI is defined by a search query that derives a value indicative of the performance of a service at a point in time or during a period of time, the value derived from machine data pertaining to one or more entities that provide the service;

assigning one or more threshold values to the thresholds, comprising, for each rime frame;

identifying training data for the time frame, and

determining the one or more threshold values for the time frame in consideration of the identified training data; and

wherein the assigning the one or more threshold values comprises assigning a first threshold value to a threshold and subsequently assigning a second threshold value to the threshold, wherein the first threshold value and the second threshold value are based on training data from different time durations;

wherein the method is performed by a computer system comprising one or more processors.

24. The non-transitory computer readable storage medium of claim 23, wherein assigning the one or more threshold values comprises assigning the one or more threshold values to the thresholds automatically based on a schedule, a frequency interval, or an event.

25. The non-transitory computer readable storage medium of claim 23, wherein determining the one or more threshold values further comprise:

analyzing the KPI values to determine a statistical metric indicating changes in the training data; and

updating the set of one or more thresholds for the one or more time frames.

26. The non-transitory computer readable storage medium of claim 23, wherein the training data comprises simulated data, historical data, or example data.

27. The non-transitory computer readable storage medium of claim 23, wherein the training data comprises simulated values, historical values, or example values of the KPI.

30. The non-transitory computer readable storage medium of claim 23, wherein the training data comprises training data that was generated by or about the one or more entities during a fixed duration of time.