Monitoring service-level performance using a key performance indicator (KPI) correlation search

• US 9,294,361 B1
• Filed: 01/31/2015
• Issued: 03/22/2016
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Choudhary, Hemendra Singh, Fletcher, Tristan Antonio, Bingham, Brian, Hsiao, Fang I., Reyes, Brian C.
• Primary Examiner(s): Maung, Zarni
• Application Number:
  US14/611,200
• Publication Number:
  US 20160105330A1
• Time in Patent Office: 416 Days
• Field of Search: 709/203, 709223-229, 709/250
• US Class Current: 1/1
• CPC Class Codes: G06F 11/321 Display for diagnostics, e.g. diagnostic result display, self-test user interface
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/903 Querying for retrieval from the web G06F16/953
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 9/542 : Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06Q 10/0637 : Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
• CPC Class Codes: G06Q 10/0639 : Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 10/06393 : Score-carding, benchmarking or key performance indicator [KPI] analysis
• CPC Class Codes: G06T 11/206 : Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: H04L 41/0213 : Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/0806 : for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/22 : comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/5009 : Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5032 : Generating service level reports
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/091 : Measuring contribution of individual network components to actual service level
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 43/55 : Testing of service level quality, e.g. simulating service usage
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]

Abstract

One or more processing devices cause display of a graphical user interface (GUI) that includes a correlation search portion that enables a user to specify information for a key performance indicator (KPI) correlation search definition. The KPI correlation search definition includes search information and trigger determination information. The search information identifies KPI values, indicative of the KPI states, in a data store. The trigger determination information includes trigger criteria. The trigger determination evaluates the identified KPI values using the trigger criteria to determine whether to cause a defined action. A contribution threshold for a particular KPI definition is received via the GUI. The contribution threshold corresponds to a particular KPI state. The contribution threshold is stored as trigger criteria information. Each of the KPI values is derived from machine data pertaining to entities identified in a service definition using a search query specified by a KPI definition for the service.

Claims

1. A method comprising:

causing display of a graphical user interface (GUI) comprising a correlation search portion, the correlation search portion enabling a user to specify information for a KPI correlation search definition, the KPI correlation search definition comprising;

information for a search identifying a plurality of KPI values in a data store, each KPI value indicative of one of a plurality of KPI states, and

information for a trigger determination including trigger criteria, the trigger determination evaluating the identified plurality of KPI values using the trigger criteria to determine whether to cause a defined action;

receiving in response to user interaction with the GUI a contribution threshold corresponding to a particular KPI definition and corresponding to a particular one of the plurality of KPI states;

storing the contribution threshold as trigger criteria information of the KPI correlation search definition in computer storage;

wherein each of the plurality of KPI values in the data store is derived by an automatically performed search query that defines a KPI associated with a service from machine data pertaining to one or more entities identified in a stored service definition for the service; and

wherein the method is performed by a computer system comprising one or more processing devices coupled to the computer storage.

2. The method of claim 1 wherein the correlation search portion presents information for each of a plurality of correlation KPI'"'"'s.

3. The method of claim 1 wherein the correlation search portion presents information for each of a plurality of correlation KPI'"'"'s, and wherein the GUI further comprises a KPI selection portion enabling a user to select the plurality of correlation KPI'"'"'s from a plurality of candidate KPI'"'"'s.

4. The method of claim 1 wherein the correlation search portion presents information for each of a plurality of correlation KPI'"'"'s, and wherein the GUI further comprises a KPI selection portion enabling the user to select the plurality of correlation KPI'"'"'s from a plurality of candidate KPI'"'"'s, and wherein the GUI further comprises a service selection portion enabling the user to select one or more services used to determine the plurality of candidate KPI'"'"'s.

5. The method of claim 1 wherein the correlation search portion presents information for each of a plurality of correlation KPI'"'"'s, and wherein the GUI further comprises a KPI selection portion enabling the user to select the plurality of correlation KPI'"'"'s from a plurality of candidate KPI'"'"'s, and wherein the GUI further comprises a service selection portion enabling the user to select one or more services used to determine the plurality of candidate KPI'"'"'s, and wherein the determination of the plurality of candidate KPI'"'"'s is based at least in part on a dependency relationship between two services.

6. The method of claim 1 wherein the correlation search portion presents information for each of a plurality of correlation KPI'"'"'s, and wherein the GUI further comprises a service selection portion enabling the user to select one or more services used to determine a plurality of candidate KPI'"'"'s from which the user is enabled to select the plurality of correlation KPI'"'"'s.

7. The method of claim 1 wherein the correlation search portion presents information for each of a plurality of correlation KPI'"'"'s, and wherein the GUI further comprises a service selection portion enabling the user to select one or more services used to determine a plurality of candidate KPI'"'"'s from which the user is enabled select the plurality of correlation KPI'"'"'s, and wherein the determination of the plurality of candidate KPI'"'"'s is based at least in part on a dependency relationship between a selected service and a second service.

8. The method of claim 1 wherein the defined action is recording a notable event comprising information identifying the KPI correlation search definition and a timestamp.

9. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes deriving a KPI state summary value from the identified plurality of KPI values that each was derived using the search query specified by a same KPI definition and that each is indicative of a same one of a plurality of KPI states.

10. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes deriving a KPI state summary value from the identified plurality of KPI values that each was derived using the search query specified by a same KPI definition and that each is indicative of a same one of a plurality of KPI states, and determining the inclusion of the KPI state summary value in a range of values having an end of the range specified by the contribution threshold.

11. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes deriving a KPI state summary value from the identified plurality of KPI values that each was derived using the search query specified by a same KPI definition and that each is indicative of a same one of a plurality of KPI states, and determining the inclusion of the KPI state summary value in a range of values having an end of the range specified by the contribution threshold, wherein the range of values defines those values indicating a KPI-state trigger contribution favoring causation of the defined action.

12. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes assessing for each correlation KPI, whether, for any particular one of the plurality of KPI states, a summary value representing the identified plurality of KPI values that each was derived by the search query that defines the particular correlation KPI and that is each indicative of the particular one of the plurality of KPI states is within a range of values having an end of the range specified by a corresponding contribution threshold and the range favoring causation of the defined action.

13. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes assessing for each correlation KPI, whether, for any particular one of the plurality of KPI states, a summary value representing the identified plurality of KPI values that each was derived using by the search query that defines specified by the KPI definition for the particular correlation KPI and that is each indicative of the particular one of the plurality of KPI states is within a range of values having an end of the range specified by a corresponding contribution threshold and the range favoring causation of the defined action;

and wherein the trigger determination comprises determining to cause the defined action in response to the production, for each correlation KPI, of a positive result by the assessing.

14. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes assessing for each correlation KPI, whether, for any particular one of the plurality of KPI states, a summary value representing the identified plurality of KPI values that each was derived by the search query that defines the particular correlation KPI and that is each indicative of the particular one of the plurality of KPI states is within a range of values having an end of the range specified by a corresponding contribution threshold and the range favoring causation of the defined action;

and wherein the trigger determination comprises determining to cause the defined action in response to the production, for each correlation KPI, of a positive result by the assessing, and causing the defined action in response to the determining to cause the defined action.

15. The method of claim 1 wherein the evaluating the identified plurality of KPI values using the trigger criteria includes assessing for each correlation KPI, whether, for any particular one of the plurality of KPI states, a summary value representing the identified plurality of KPI values that each was derived by the search query that defines the particular correlation KPI and that is each indicative of the particular one of the plurality of KPI states is within a range of values having an end of the range specified by a corresponding contribution threshold and the range favoring causation of the defined action;

and wherein the trigger determination comprises determining to cause the defined action in response to the production of a positive result by the assessing for each correlation KPI, and causing the defined action in response to the determining to cause the defined action, wherein the defined action is recording a notable event comprising information identifying the KPI correlation search definition and a timestamp.

16. The method of claim 1 further comprising:

receiving in response to user interaction with the GUI a time interval specification; and

storing the time interval specification as search information of the KPI correlation search definition.

17. The method of claim 1 further comprising:

receiving in response to user interaction with the GUI a time frequency specification; and

storing the time frequency specification in the KPI correlation search definition.

18. The method of claim 1 further comprising:

receiving in response to user interaction with the GUI a severity specification for association with a notable event record produced by the defined action; and

storing the severity specification in the KPI correlation search definition.

19. The method of claim 1 further comprising:

receiving in response to user interaction with the GUI a severity specification for association with a notable event record produced by the defined action; and

storing the severity specification in the KPI correlation search definition; and

wherein the severity specification comprises a fixed value or a specification for determining a dynamic value.

20. The method of claim 1 wherein the correlation search portion of the GUI comprises an interactive element operatively coupled to a displayed value for the contribution threshold and to a visual element depicting the displayed value for the contribution threshold, the visual element appearing in conjunction with a second visual element depicting a value derived from KPI values in a data store corresponding to the particular KPI definition of the contribution threshold and to the particular one of the plurality of KPI states of the contribution threshold.

21. The method of claim 1 wherein the correlation search portion of the GUI comprises an interactive element operatively coupled to a displayed value for the contribution threshold and to a visual element depicting the displayed value for the contribution threshold, the visual element appearing in conjunction with a second visual element depicting a value derived from KPI values in a data store corresponding to the particular KPI definition of the contribution threshold and to the particular one of the plurality of KPI states of the contribution threshold, and wherein the interactive element is control handle for a slider bar.

22. The method of claim 1, wherein the machine data associated with a particular entity is derived from different sources.

23. The method of claim 1, wherein the machine data associated with a particular entity includes data in different formats.

24. The method of claim 1, wherein the machine data associated with a particular entity includes log data produced by the entity.

25. The method of claim 1, wherein the machine data associated with a particular entity is obtained through an application programming interface (API) from software that monitors the performance of the entity.

26. The method of claim 1, wherein the machine data associated with a particular entity is derived from network packet data that referenced the entity.

27. The method of claim 1, wherein the machine data associated with a particular entity is represented as events comprising a portion of raw data.

28. The method of claim 1, wherein an entity includes at least one of a host machine, a virtual machine, a switch, a firewall, a router, and a sensor.

29. A system comprising:

a memory; and

a processing device coupled with the memory to;

cause display of a graphical user interface (GUI) comprising a correlation search portion, the correlation search portion enabling a user to specify information for a KPI correlation search definition, the KPI correlation search definition comprising;

information for a search identifying a plurality of KPI values in a data store, each KPI value indicative of one of a plurality of KPI states, and

information for a trigger determination including trigger criteria, the trigger determination evaluating the identified plurality of KPI values using the trigger criteria to determine whether to cause a defined action;

receive in response to user interaction with the GUI a contribution threshold corresponding to a particular KPI definition and corresponding to a particular one of the plurality of KPI states;

store the contribution threshold as trigger criteria information of the KPI correlation search definition in computer storage;

wherein each of the plurality of KPI values in the data store is derived by an automatically performed search query that defines a KPI associated with a service from machine data pertaining to one or more entities identified in a stored service definition for the service.

30. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by one or more processing devices, cause the processing device to perform operations comprising:

causing display of a graphical user interface (GUI) comprising a correlation search portion, the correlation search portion enabling a user to specify information for a KPI correlation search definition, the KPI correlation search definition comprising;

information for a search identifying a plurality of KPI values in a data store, each KPI value indicative of one of a plurality of KPI states, and

information for a trigger determination including trigger criteria, the trigger determination evaluating the identified plurality of KPI values using the trigger criteria to determine whether to cause a defined action;

receiving in response to user interaction with the GUI a contribution threshold corresponding to a particular KPI definition and corresponding to a particular one of the plurality of KPI states;

storing the contribution threshold as trigger criteria information of the KPI correlation search definition in computer storage;

wherein each of the plurality of KPI values in the data store is derived by an automatically performed search query that defines a KPI associated with a service from machine data pertaining to one or more entities identified in a stored service definition for the service.