Adjusting weights for aggregated key performance indicators that include a graphical control element of a graphical user interface

• US 10,572,541 B2
• Filed: 04/18/2018
• Issued: 02/25/2020
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Fletcher, Tristan Antonio, Choudhary, Hemendra Singh, Sharp, Clint
• Primary Examiner(s): Lim, Krisna
• Application Number:
  US15/955,990
• Publication Number:
  US 20180241660A1
• Time in Patent Office: 678 Days
• Field of Search: 709224
• CPC Class Codes: G06F 11/321 Display for diagnostics, e.g. diagnostic result display, self-test user interface
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/903 Querying for retrieval from the web G06F16/953
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 9/542 : Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06Q 10/0637 : Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
• CPC Class Codes: G06Q 10/0639 : Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 10/06393 : Score-carding, benchmarking or key performance indicator [KPI] analysis
• CPC Class Codes: G06T 11/206 : Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: H04L 41/0213 : Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/0806 : for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/22 : comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/5009 : Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5032 : Generating service level reports
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/091 : Measuring contribution of individual network components to actual service level
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 43/55 : Testing of service level quality, e.g. simulating service usage
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]

Abstract

Techniques are disclosed for providing an aggregate key performance indicator (KPI) that spans multiple services and for providing adjustment to KPI factors via a GUI that enables an aggregate KPI to be configured with feedback that better characterizes the performance of the services. The GUI may enable selection of KPIs and adjustment to weights (e.g., importance) associated with the KPIs. The weight of a KPI may affect the influence a value of the KPI has on the calculation of an aggregate KPI value (e.g., score). The GUI may provide near real-time feedback concerning the effect the weights have on the aggregate KPI value by displaying the aggregate KPI value (e.g., score) and updating the aggregate KPI value as the weights are adjusted.

Claims

1. A method comprising:

causing for display a graphical user interface that displays key performance indicators (KPIs) and graphical control elements for the KPIs, the KPIs being associated with one or more services, the graphical control elements enabling adjustment of a weight of one of the KPIs;

causing for display, in the graphical user interface, a value of an aggregate KPI that is determined in view of weights and values of one or more of the KPIs associated with the services;

in response to receiving an indication of an adjustment of the weight of one of the KPIs, modifying the value of the aggregate KPI in the graphical user interface to reflect the adjusted weight, wherein each of the KPIs is defined by a search query of machine data and indicates an aspect of how a service provided by one or more entities is performing at a point in time or during a period of time thereby transforming machine data to the KPI indicating the aspect of how the service is performing;

wherein each entity of the one or more entities corresponds to an entity definition having an identification of machine data from or about the entity;

wherein a service of the one or more of services is represented by a service definition that references the entity definition; and

wherein the method is performed by one or more processing devices.

2. The method of claim 1, wherein the search query derives a value for a respective KPI from machine data produced by the one or more entities that provide the one or more services.

3. The method of claim 1, wherein the value of the aggregate KPI is calculated using a weighted average of values from KPIs of multiple services.

4. The method of claim 1, wherein the value of the one or more KPIs is determined by retrieving a most recent value for each of a plurality of KPIs from a data store, wherein the most recent value for a first KPI and the most recent value for a second KPI are derived from different time periods.

5. The method of claim 1, wherein the value of the one or more KPIs is derived by executing the search query defining each of the one or more KPIs.

6. The method of claim 1, wherein the graphical control element to enable adjustment of a weight of a respective KPI to an exclusion value that causes the respective KPI to be excluded from a calculation of the value of the aggregate KPI.

7. The method of claim 1, wherein the graphical control element to enable adjustment of a weight of a respective KPI to an exclusion value that causes the respective KPI to be excluded from a calculation of the value of the aggregate KPI, wherein the exclusion value is a minimum value associated with a range of weighting values.

8. The method of claim 1, wherein the graphical control element to enable adjustment of a weight of a respective KPI to a priority value that causes the respective KPI to override other KPIs when calculating the value of the aggregate KPI, wherein the value of the aggregate KPI is calculated based on only one of the KPIs having the weighting value.

9. The method of claim 1, wherein the graphical control element to enable adjustment of a weight of a respective KPI to a priority value that causes the respective KPI to override other KPIs when calculating the value of the aggregate KPI, wherein the value of the aggregate KPI is calculated based on only one of the KPIs having the weighting value, wherein the priority value is a maximum value associated with a range of weighting values.

10. The method of claim 1, further comprising:

comparing the value for the aggregate KPI to a threshold; and

causing generation of an alert based on the comparing.

11. The method of claim 1, further comprising:

receiving another indication to generate an alert when the value of the aggregate KPI exceeds a threshold associated with a critical state;

generating a correlation search based on the plurality of KPIs and weights associated with the plurality of KPIs; and

scheduling the correlation search to periodically execute.

12. The method of claim 1, further comprising:

receiving a first selection of services in a first display component of the graphical user interface, the first display component enabling selection of a subset of services from a list of services within an IT environment;

in response to the first selection, causing for display in a second display component of the graphical user interface a list of the KPIs associated with the one or more services, wherein the second display component enables selection of a subset of KPIs from the list of KPIs;

receiving a second selection of the subset of KPIs from the list of KPIs in the second display component of the graphical user interface; and

in response to the second selection, causing for display in a third display component of the graphical user interface the one or more KPIs and the graphical control elements for the KPIs, wherein the third display component enables adjustment of weights for the subset of KPIs.

13. The method of claim 1, wherein the graphical control element comprises a slider.

14. The method of claim 1, wherein the graphical user interface provides a visual indication of a state corresponding to the value of the aggregate KPI.

15. The method of claim 1, wherein the graphical user interface provides a visual indication of a state corresponding to the value of the aggregate KPI, wherein the state is critical when the value of the aggregate KPI exceeds a threshold value.

16. The method of claim 1, wherein the graphical user interface provides a visual indication of a state corresponding to each of the KPIs, wherein the state of a respective KPI is critical when the value of the respective KPI exceeds a threshold value.

17. The method of claim 1, wherein the search query derives the value for the KPI using a late-binding schema to extract an initial value from machine data and then performs a calculation with the initial value.

18. The method of claim 1, wherein the search query derives the value for the KPI by applying a late-binding schema to events containing raw portions of the machine data.

19. A system comprising:

a memory; and

a processing device coupled with the memory to;

cause for display a graphical user interface that displays key performance indicators (KPIs) and graphical control elements for the KPIs, the KPIs being associated with one or more services, the graphical control elements enabling adjustment of a weight of one of the KPIs;

cause for display, in the graphical user interface, a value of an aggregate KPI that is determined in view of weights and values of one or more of the KPIs associated with the services;

in response to receiving an indication of an adjustment of the weight of one of the KPIs, modifying the value of the aggregate KPI in the graphical user interface to reflect the adjusted weight, wherein each of the KPIs is defined by a search query of machine data and indicates an aspect of how a service provided by one or more entities is performing at a point in time or during a period of time thereby transforming machine data to the KPI indicating the aspect of how the service is performing;

wherein each entity of the one or more entities corresponds to an entity definition having an identification of machine data from or about the entity; and

wherein a service of the one or more services is represented by a service definition that references the entity definition.

20. The system of claim 19, wherein the search query derives a value for a respective KPI from machine data produced by the one or more entities that provide the one or more services.

21. The system of claim 19, wherein the value of the aggregate KPI is calculated using a weighted average of values from KPIs of multiple services.

22. The system of claim 19, wherein the value of the one or more KPIs is determined by retrieving a most recent value for each of a plurality of KPIs from a data store, wherein the most recent value for a first KPI and the most recent value for a second KPI are derived from different time periods.

23. The system of claim 19, wherein the value of the one or more KPIs is derived by executing the search query defining each of the one or more KPIs.

24. The system of claim 19, wherein the graphical control element to enable adjustment of a weight of a respective KPI to an exclusion value that causes the respective KPI to be excluded from a calculation of the value of the aggregate KPI.

25. The system of claim 19, wherein the graphical control element to enable adjustment of a weight of a respective KPI to an exclusion value that causes the respective KPI to be excluded from a calculation of the value of the aggregate KPI, wherein the exclusion value is a minimum value associated with a range of weighting values.

26. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by a processing device, cause the processing device to perform operations comprising:

causing for display a graphical user interface that displays key performance indicators (KPIs) and graphical control elements for the KPIs, the KPIs being associated with one or more services, the graphical control elements enabling adjustment of a weight of one of the KPIs;

causing for display, in the graphical user interface, a value of an aggregate KPI that is determined in view of weights and values of one or more of the KPIs associated with the services;

in response to receiving an indication of an adjustment of the weight of one of the KPIs, modifying the value of the aggregate KPI in the graphical user interface to reflect the adjusted weight, wherein each of the KPIs is defined by a search query of machine data and indicates an aspect of how a service provided by one or more entities is performing at a point in time or during a period of time thereby transforming machine data to the KPI indicating the aspect of how the service is performing;

wherein each entity of the one or more entities corresponds to an entity definition having an identification of machine data from or about the entity;

wherein a service of the one or more services is represented by a service definition that references the entity definition; and

wherein the method is performed by one or more processing devices.

27. The non-transitory computer readable storage medium of claim 26, wherein the search query derives a value for a respective KPI from machine data produced by the one or more entities that provide the one or more services, and wherein the value of the one or more KPIs is derived by executing the search query defining each of the one or more KPIs.

28. The non-transitory computer readable storage medium of claim 26, wherein the value of the aggregate KPI is calculated using a weighted average of values from KPIs of multiple services.

29. The non-transitory computer readable storage medium of claim 26, wherein the value of the one or more KPIs is determined by retrieving a most recent value for each of a plurality of KPIs from a data store, wherein the most recent value for a first KPI and the most recent value for a second KPI are derived from different time periods.

30. The non-transitory computer readable storage medium of claim 26, wherein the graphical control element to enable adjustment of a weight of a respective KPI to an exclusion value that causes the respective KPI to be excluded from a calculation of the value of the aggregate KPI.