Correlating performance data and log data using diverse data stores

• US 10,592,522 B2
• Filed: 06/05/2018
• Issued: 03/17/2020
• Est. Priority Date: 04/30/2013
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Bingham, Brian, Fletcher, Tristan, Bhide, Alok Anant
• Primary Examiner(s): Sun, Charlie
• Application Number:
  US16/000,745
• Publication Number:
  US 20180307734A1
• Time in Patent Office: 651 Days
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

The disclosed system and method acquire and store performance measurements relating to performance of a component in an information technology (IT) environment and log data produced by the IT environment, in association with corresponding time stamps. The disclosed system and method correlate at least one of the performance measurements with at least one of the portions of log data.

Claims

1. A method comprising:

acquiring, by a computer system, a plurality of performance measurements for a performance metric associated with at least one hardware or software component of an information technology (IT) environment;

storing, by the computer system, the acquired performance measurements in a first data store;

acquiring, by the computer system, a plurality of portions of log data representing activity of at least one hardware or software component of the IT environment;

storing, by the computer system, the acquired portions of log data in a second data store separate from the first data store; and

correlating, by the computer system, at least one of the stored performance measurements with at least one of the stored portions of log data, wherein said correlating includes;

applying a search query having correlation criterion to the stored performance measurements in the first data store and the stored portions of log data in the second data store separate from the first data store, and

causing display of an indication of a performance measurement that satisfies a correlation criterion and causing display of an indication of a portion of log data that satisfies the correlation criterion.

2. A method as recited in claim 1, wherein applying the search query includes searching the first data store for performance data that satisfy the correlation criterion and searching the second data store for log data that satisfy the correlation criterion.

3. A method as recited in claim 1, further comprising:

obtaining the search query in a text-based search query language, the search query specified by a user;

wherein applying the search query includes, in response to the search query, searching the first data store for performance data that satisfy the correlation criterion and searching the second data store for log data that satisfy the correlation criterion.

4. A method as recited in claim 1, further comprising:

obtaining the search query in a text-based search query language, the search query including the correlation criterion, wherein the correlation criterion comprises a user-specified value or a user-specified range of values, for a user-specified field;

wherein applying the search query includes, in response to the search query, searching the first data store for performance data that satisfy the correlation criterion and searching the second data store for log data that satisfy the correlation criterion.

5. A method as recited in claim 1, wherein the correlation criterion is a user-specified value or a user-specified range of values, for a user-specified field.

6. A method as recited in claim 1, wherein the performance measurements are stored in a time-series data store in a first format, and the portions of log data are stored in said time-series data store in a second format different from the first format.

7. A method as recited in claim 1, wherein the performance measurements are stored in the first data store in a first format, and the portions of log data are stored in the second data store in a second format different from the first format.

8. A method as recited in claim 1, wherein the performance measurements are stored in the first data store in a first format, and the portions of log data are stored in the second data store in a second format different from the first format;

the method further comprising obtaining the search query in a text-based search query language, the search query including the correlation criterion;

wherein applying the search query includes, in response to the search query, searching the first data store for performance data that satisfy the correlation criterion and searching the second data store for log data that satisfy the correlation criterion.

9. A method as recited in claim 1, wherein the performance measurements are stored in the first data store in a first format, and the portions of log data are stored in the second data store in a second format different from the first format;

the method further comprising obtaining the search query, wherein the correlation criterion comprises a user-specified value or a user-specified range of values, for a user-specified field;

wherein applying the search query includes, in response to the search query, searching the first data store for performance data that satisfy the correlation criterion and searching the second data store for log data that satisfy the correlation criterion.

10. A method as recited in claim 1, further comprising:

causing display of a graphical user interface element, wherein the graphical user interface element is one of a drop-down selection list, a slider or a checkbox; and

obtaining the correlation criterion from a user via the graphical user interface element.

11. A method as recited in claim 1, further comprising:

causing display of a graphical user interface element;

obtaining the correlation criterion from a user via the graphical user interface element, the correlation criterion for use by the computer system as a portion of a search query; and

generating, by the computer system, the search query in a search query language, the search query including the correlation criterion.

12. A method as recited in claim 1, further comprising:

causing display of a graphical user interface element;

obtaining the correlation criterion from a user via the graphical user interface element, the correlation criterion for use by the computer system as a portion of a search query; and

generating, by the computer system, the search query in a search query language, the search query including the correlation criterion.

13. A method as recited in claim 1, wherein the correlation criterion identifies a hardware or software component in the IT environment.

14. A method as recited in claim 1, wherein each of the acquired performance measurements and each of the acquired portions of log data is stored with a time-stamp.

15. A method as recited in claim 1, wherein:

each of the acquired performance measurements and each of the acquired portions of log data is stored with a time-stamp; and

said correlating comprises identifying at least one of the stored performance measurements and at least one of the stored portions of log data that have time stamps that satisfy a user-specified time criterion.

16. A method as recited in claim 1, wherein said storing comprises:

storing the acquired performance measurements as time-stamped performance events and storing the acquired portions of log data as time-stamped log events, such that each performance event and each log event is stored in association with a respective time stamp.

17. A method as recited in claim 1, wherein the indication of the performance measurement that satisfies the correlation criterion and the indication of the portion of log data that satisfies the correlation criteria are concurrently displayed.

18. A method as recited in claim 1, wherein the plurality of performance measurements have been determined by direct measurement of a hardware or software component in the IT environment.

19. A method as recited in claim 1, wherein the plurality of portions of log data are from a text-based log file.

20. A method as recited in claim 1, wherein the plurality of performance measurements are not derived from a log file and are acquired independently of the plurality of portions of log data.

21. A method as recited in claim 1, wherein the plurality of performance measurements are acquired independently of the plurality of portions of log data, and wherein the performance measurements have been determined by direct measurement of a hardware or software component in the IT environment and the plurality of portions of log data are from a text-based log file.

22. A method as recited in claim 1, wherein the plurality of performance measurements are acquired independently of the plurality of portions of log data by direct measurement of a hardware or software component in the IT environment, and the plurality of portions of log data are acquired independently of the performance measurements.

23. A method as recited in claim 1, wherein acquiring the plurality of performance measurements comprises acquiring, via an application programming interface (API), the plurality of performance measurements from a third-party software application that collects the performance measurements.

24. A method as recited in claim 1, further comprising:

acquiring structure data indicative of structure characteristics of the IT environment;

storing the acquired structure data indicative of structure characteristics of the IT environment; and

correlating a performance characteristic of the IT environment with a structure characteristic of the IT environment, based on the stored performance measurements and stored structure data.

25. A method as recited in claim 1, further comprising:

acquiring structure data indicative of structure characteristics of the IT environment, wherein the structure data is derived from log data from the IT environment;

storing the acquired structure data indicative of structure characteristics of the IT environment; and

correlating a performance characteristic of the IT environment with a structure characteristic of the IT environment, based on the stored performance measurements and stored structure data.

26. A method as recited in claim 1, wherein the performance metric comprises a performance metric for at least one hardware or software resource of a computer system.

27. A method as recited in claim 1, wherein the performance metric comprises a performance metric for at least one virtual machine or virtual machine host.

28. A method as recited in claim 1, wherein the performance metric comprises a performance metric for a virtual machine cluster.

29. A non-transitory machine-readable storage medium for use in a processing system of a data intake and query system, the non-transitory machine-readable storage medium storing instructions, an execution of which in the processing system causes the processing system to perform operations comprising:

acquiring, by a computer system, a plurality of performance measurements for a performance metric associated with at least one hardware or software component of an information technology (IT) environment;

storing, by the computer system, the acquired performance measurements in a first data store;

acquiring, by the computer system, a plurality of portions of log data representing activity of at least one hardware or software component of the IT environment;

storing, by the computer system, the acquired portions of log data in a second data store separate from the first data store; and

correlating, by the computer system, at least one of the stored performance measurements with at least one of the stored portions of log data, wherein said correlating includes;

applying a search query having correlation criterion to the stored performance measurements in the first data store and the stored portions of log data in the second data store separate from the first data store, and

causing display of an indication of a performance measurement that satisfies a correlation criterion and causing display of an indication of a portion of log data that satisfies the correlation criterion.

30. A system comprising:

a communication device through which to communicate on a computer network; and

at least one processor operatively coupled to the communication device and configured to perform operations including

acquiring, by a computer system, a plurality of performance measurements for a performance metric associated with at least one hardware or software component of an information technology (IT) environment;

storing, by the computer system, the acquired performance measurements in a first data store;

acquiring, by the computer system, a plurality of portions of log data representing activity of at least one hardware or software component of the IT environment;

storing, by the computer system, the acquired portions of log data in a second data store separate from the first data store; and

correlating, by the computer system, at least one of the stored performance measurements with at least one of the stored portions of log data, wherein said correlating includes;

applying a search query having correlation criterion to the stored performance measurements in the first data store and the stored portions of log data in the second data store separate from the first data store, and

causing display of an indication of a performance measurement that satisfies a correlation criterion and causing display of an indication of a portion of log data that satisfies the correlation criterion.