Monitoring overall service-level performance using an aggregate key performance indicator derived from machine data

• US 9,584,374 B2
• Filed: 10/30/2014
• Issued: 02/28/2017
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Bingham, Brian John, Fletcher, Tristan Antonio, Choudhary, Hemendra Singh
• Primary Examiner(s): BLAIR, DOUGLAS B
• Application Number:
  US14/528,906
• Publication Number:
  US 20160103907A1
• Time in Patent Office: 852 Days
• Field of Search: 709/224
• US Class Current: 1/1
• CPC Class Codes: G06F 11/321 Display for diagnostics, e.g. diagnostic result display, self-test user interface
• CPC Class Codes: G06F 11/34 Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/903 Querying for retrieval from the web G06F16/953
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 : Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 9/542 : Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: G06Q 10/0637 : Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
• CPC Class Codes: G06Q 10/0639 : Performance analysis of employees; Performance analysis of enterprise or organisation operations
• CPC Class Codes: G06Q 10/06393 : Score-carding, benchmarking or key performance indicator [KPI] analysis
• CPC Class Codes: G06T 11/206 : Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: H04L 41/0213 : Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/0806 : for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/22 : comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/5009 : Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 41/5032 : Generating service level reports
• CPC Class Codes: H04L 43/04 : Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 : for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/091 : Measuring contribution of individual network components to actual service level
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 43/55 : Testing of service level quality, e.g. simulating service usage
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 69/329 : in the application layer [OSI layer 7]

Abstract

One or more processing devices derive a value for each of a plurality of key performance indicators (KPIs). Each KPI indicates a different aspect of how the same service provided by one or more entities is performing at a point in time. Each KPI is defined by a search query that derives the value for that KPI from machine data associated with the one or more entities that provide the same service. The one or more processing devices calculate a value for an aggregate KPI for the same service from the values for each of the plurality of KPIs.

Claims

1. A method, comprising:

deriving a value for each of a plurality of key performance indicators (KPIs) by executing for each KPI, automatically on a time basis, a search query that defines the KPI and that derives the value for that KPI from machine data identified in one or more stored entity definitions referenced in a stored service definition, each KPI indicating a different aspect of how a same service represented by the service definition and provided by one or more entities represented by the entity definitions is performing at a point in time or during a period of time;

mapping the value for each of the plurality of KPIs to one of a plurality of states, each state defined by a range of values;

assigning a weighting to each of the plurality of KPIs based on the state to which the value for that KPI has been mapped;

calculating a value for an aggregate KPI for the same service from the values for each of the plurality of KPIs, including applying the assigned weightings for each of the plurality of KPIs to the respective values for each of the plurality of KPIs;

wherein the machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment, the machine data received over a network and reflected in computer storage in representations of events each having a segment of the machine data; and

wherein the method is performed by a computing system, the computing system comprising one or more processing devices coupled to computer memory for storing the service definition, entity definitions, and at least one search query defining at least one KPI.

2. The method of claim 1, further comprising:

comparing the value for the aggregate KPI to a threshold; and

causing generation of an alert based on the comparison.

3. The method of claim 1, further comprising:

comparing the value for the aggregate KPI to a threshold; and

generating a notable event based on the comparison.

4. The method of claim 1, further comprising:

comparing the value for the aggregate KPI to a threshold; and

generating an entry in an incident-review dashboard based on the comparison.

5. The method of claim 1, further comprising:

comparing the value for the aggregate KPI to a threshold; and

causing a user-specified action based on the comparison.

6. The method of claim 1, wherein the value of the aggregate KPI is periodically updated.

7. The method of claim 1, wherein the machine data includes at least one from among unstructured data, log data, and wire data.

8. The method of claim 1, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

9. The method of claim 1, wherein the search query defining a KPI derives the value for that KPI in part by applying a late-binding schema to machine data.

10. The method of claim 1, wherein the search query defining a KPI derives the value for that KPI in part by applying a late-binding schema to events containing raw portions of the machine data.

20. The method of claim 6, further comprising:

comparing the value for the aggregate KPI to a threshold; and

at least one from among;

causing generation of an alert based on the comparison,

generating a notable event based on the comparison,

generating an entry in an incident-review dashboard based on the comparison, and

causing a user-specified action based on the comparison.

11. A method comprising:

deriving a value for each of a plurality of key performance indicators (KPIs) by executing for each KPI, automatically on a time basis, a search query that defines the KPI and that derives the value for that KPI from machine data identified in one or more stored entity definitions referenced in a stored service definition, each KPI indicating a different aspect of how a same service represented by the service definition and provided by one or more entities represented by the entity definitions is performing at a point in time or during a period of time;

mapping the value for each of the plurality of KPIs to one of a plurality of states, each state defined by a range of values;

assigning a rating to each of the plurality of KPIs based on the state to which the value for that KPI has been mapped, the rating to be assigned based on particular state mapping being user entered;

calculating a value for an aggregate KPI for the same service, including mapping the state-based rating to the values for each of the plurality of KPIs;

wherein the machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment, the machine data received over a network and reflected in computer storage in representations of events each having a segment of the machine data; and

wherein the method is performed by a computing system, the computing system comprising one or more processing devices coupled to computer memory for storing the service definition, entity definitions, and at least one search query defining at least one KPI.

12. The method of claim 11, further comprising:

comparing the value for the aggregate KPI to a threshold; and

at least one from among;

causing generation of an alert based on the comparison,

generating a notable event based on the comparison,

generating an entry in an incident-review dashboard based on the comparison, and

causing a user-specified action based on the comparison.

13. The method of claim 11, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

14. The method of claim 11, wherein the search query defining a KPI derives the value for that KPI in part by applying a late-binding schema to events containing raw portions of the machine data.

15. A method comprising:

deriving a value for each of a plurality of key performance indicators (KPIs) by executing for each KPI, automatically on a time basis, a search query that defines the KPI and that derives the value for that KPI from machine data identified in one or more stored entity definitions referenced in a stored service definition, each KPI indicating a different aspect of how a same service represented by the service definition and provided by one or more entities represented by the entity definitions is performing at a point in time or during a period of time;

receiving a user-selected weighting for each of the plurality of KPIs,

receiving a user-selected rating for each of a plurality of states to which a derived value for a KPI can be mapped;

calculating a value for an aggregate KPI for the same service from the values for each of the plurality of KPIs, including applying to the derived value for each of the plurality of KPIs a weighting based on both the user-selected weighting for the KPI to which the derived value corresponds and the user-selected rating for the state to which the derived value maps;

wherein the machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment, the machine data received over a network and reflected in computer storage in representations of events each having a segment of the machine data; and

wherein the method is performed by a computing system, the computing system comprising one or more processing devices coupled to computer memory for storing the service definition, entity definitions, and at least one search query defining at least one KPI.

16. The method of claim 15, further comprising:

comparing the value for the aggregate KPI to a threshold; and

at least one from among;

causing generation of an alert based on the comparison,

generating a notable event based on the comparison,

generating an entry in an incident-review dashboard based on the comparison, and

causing a user-specified action based on the comparison.

17. The method of claim 15, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

18. The method of claim 15, wherein the search query defining a KPI derives the value for that KPI in part by applying a late-binding schema to events containing raw portions of the machine data.

19. A method comprising:

deriving a value for each of a plurality of key performance indicators (KPIs) by executing for each KPI, automatically on a time basis, a search query that defines the KPI and that derives the value for that KPI from machine data identified in one or more stored entity definitions referenced in a stored service definition, each KPI indicating a different aspect of how a same service represented by the service definition and provided by one or more entities represented by the entity definitions is performing at a point in time or during a period of time;

accessing in stored memory a weighting for each of the plurality of KPIs,

accessing in stored memory a rating for each of a plurality of states to which a derived value for a KPI can be mapped;

calculating a value for an aggregate KPI for the same service from the values for each of the plurality of KPIs, including applying to the derived value for each of the plurality of KPIs a weighting based on both the weighting for the KPI to which the value corresponds and the rating for the state to which the derived value maps;

wherein the machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment, the machine data received over a network and reflected in computer storage in representations of events each having a segment of the machine data; and

wherein the method is performed by a computing system, the computing system comprising one or more processing devices coupled to computer memory for storing the service definition, entity definitions, and at least one search query defining at least one KPI.

21. The method of claim 19, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

22. The method of claim 19, wherein the search query defining a KPI derives the value for that KPI in part by applying a late-binding schema to events containing raw portions of the machine data.

23. A system comprising:

a memory; and

a processing device coupled with the memory with programming to;

derive a value for each of a plurality of key performance indicators (KPIs) by executing for each KPI, automatically on a time basis, a search query that defines the KPI and that derives the value for that KPI from machine data identified in one or more stored entity definitions referenced in a stored service definition, each KPI indicating a different aspect of how a same service represented by the service definition and provided by one or more entities represented by the entity definitions is performing at a point in time or during a period of time;

receive a user-selected weighting for each of the plurality of KPIs,

receive a user-selected rating for each of a plurality of states to which a derived value for a KPI can be mapped;

calculate a value for an aggregate KPI for the same service from the values for each of the plurality of KPIs, including applying to the derived value for each of the plurality of KPIs a weighting based on both the user-selected weighting for the KPI to which the derived value corresponds and the user-selected rating for the state to which the derived value maps; and

wherein the machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment, the machine data received over a network and reflected in computer storage in representations of events each having a segment of the machine data.

24. The system of claim 23, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

25. The system of claim 23, wherein the processing device is further to:

compare the value for the aggregate KPI to a threshold; and

at least one from among;

cause generation of an alert based on the comparison,

generate a notable event based on the comparison,

generate an entry in an incident-review dashboard based on the comparison, and

cause a user-specified action based on the comparison.

26. The system of claim 23, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

27. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by one or more processing devices, cause the processing device to perform operations comprising:

deriving a value for each of a plurality of key performance indicators (KPIs) by executing for each KPI, automatically on a time basis, a search query that defines the KPI and that derives the value for that KPI from machine data identified in one or more stored entity definitions referenced in a stored service definition, each KPI indicating a different aspect of how a same service represented by the service definition and provided by one or more entities represented by the entity definitions is performing at a point in time or during a period of time;

map the value for each of the plurality of KPIs to one of a plurality of states, each state defined by a range of values;

assign a weighting to each of the plurality of KPIs based on the state to which the value for that KPI has been mapped;

calculating a value for an aggregate KPI for the same service from the values for each of the plurality of KPIs, including applying the assigned weightings for each of the plurality of KPIs to the respective values for each of the plurality of KPIs; and

wherein the machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment, the machine data received over a network and reflected in computer storage in representations of events each having a segment of the machine data.

28. The non-transitory computer readable storage medium of claim 27, with further operations comprising:

comparing the value for the aggregate KPI to a threshold; and

at least one from among;

causing generation of an alert based on the comparison,

generating a notable event based on the comparison,

generating an entry in an incident-review dashboard based on the comparison, and

causing a user-specified action based on the comparison.

29. The non-transitory computer readable storage medium of claim 27, wherein the machine data identified in at least one entity definition includes data that comes from at least two different sources having heterogeneous formats.

30. The non-transitory computer readable storage medium of claim 27, wherein the search query defining a KPI derives the value for that KPI in part by applying a late-binding schema to events containing raw portions of the machine data.