EVENT FIELD DISTRIBUTED SEARCH DISPLAY

• US 20150058375A1
• Filed: 10/31/2014
• Published: 02/26/2015
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Zhang, Steve Yu, Sorkin, Stephen P.
• Granted Patent: US 9,129,028 B2
• Time in Patent Office: Days
• US Class Current: 707/770
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/90328 using search space presentation or visualization, e.g. category or range presentation and selection
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a “divide and conquer” algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

Claims

1. A method, comprising:

receiving a query including a criterion for searching a set of events stored across a plurality of distributed machines, wherein each distributed machine has responsibility to search a time-based subgroup of the set of events, wherein each distributed machine has responsibility to search a time-based subgroup different than time-based subgroups that other distributed machines have responsibility to search, and wherein each event in the set of events is associated with a timestamp;

in response to receiving the query, directing at least a portion of the plurality of distributed machines to search, in respective time-based subgroups, for events responsive to the query;

receiving from the distributed machines information relating to values for a field that are in the events responsive to the query;

aggregating the received information to determine a number corresponding to how many unique values exist for the field in the events responsive to the query;

displaying the number corresponding to how many unique values exist for the field;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the events include machine data.

3. The method of claim 1, wherein the events include log data.

4. The method of claim 1, wherein the events include unstructured data.

5. The method of claim 1, wherein the criterion is an evaluation of values for the field.

6. The method of claim 1, wherein the method further comprises:

for each event in at least a portion of the events responsive to the query, displaying its value for the field.

7. The method of claim 1, further comprising:

receiving from the distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query; and

displaying at least a portion of the information about the events responsive to the query.

8. The method of claim 1, further comprising:

receiving from the at least a portion of the plurality of distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query;

receiving timestamps associated with the events to which the information pertains;

displaying at least a portion of the information about the events responsive to the query in an order corresponding to the timestamps.

9. The method of claim 1, further comprising:

receiving one or more event identifiers, each event identifier corresponding to an event in the events responsive to the query;

retrieving from the distributed machines further information about the event corresponding to the event identifier using one of the event identifiers.

10. The method of claim 1, further comprising:

displaying a histogram of the events responsive to the query, wherein the histogram indicates how many of the events responsive to the query have a timestamp falling within each of a plurality of time ranges.

11. The method of claim 1, wherein the criterion for searching the set of events includes a time range to be compared to the timestamps of the events.

12. The method of claim 1, wherein the criterion for searching the set of events includes a value, set of values, or range of values for a field.

13. The method of claim 1, wherein the criterion for searching the set of events includes a specified keyword.

14. An apparatus, comprising:

a subsystem, implemented at least partially in hardware, that receives a query including a criterion for searching a set of events stored across a plurality of distributed machines, wherein each distributed machine has responsibility to search a time-based subgroup of the set of events, wherein each distributed machine has responsibility to search a time-based subgroup different than time-based subgroups that other distributed machines have responsibility to search, and wherein each event in the set of events is associated with a timestamp;

a subsystem, implemented at least partially in hardware, that, in response to receiving the query, directs at least a portion of the plurality of distributed machines to search, in respective time-based subgroups, for events responsive to the query;

a subsystem, implemented at least partially in hardware, that receives from the distributed machines information relating to values for a field that are in the events responsive to the query;

a subsystem, implemented at least partially in hardware, that aggregates the received information to determine a number corresponding to how many unique values exist for the field in the events responsive to the query;

a subsystem, implemented at least partially in hardware, that displays the number corresponding to how many unique values exist for the field.

15. The apparatus as recited in claim 14, wherein the events include machine data.

16. The apparatus as recited in claim 14, wherein the events include log data.

17. The apparatus as recited in claim 14, wherein the events include unstructured data.

18. The apparatus as recited in claim 14, wherein the criterion is an evaluation of values for the field

19. The apparatus as recited in claim 14, further comprising:

a subsystem, implemented at least partially in hardware, that, for each event in at least a portion of the events responsive to the query, displays its value for the field

20. The apparatus as recited in claim 14, further comprising:

a subsystem, implemented at least partially in hardware, that receives from the distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query; and

a subsystem, implemented at least partially in hardware, that displays at least a portion of the information about the events responsive to the query.

21. The apparatus as recited in claim 14, further comprising:

a subsystem, implemented at least partially in hardware, that receives from the at least a portion of the plurality of distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query;

a subsystem, implemented at least partially in hardware, that receives timestamps associated with the events to which the information pertains;

a subsystem, implemented at least partially in hardware, that displays at least a portion of the information about the events responsive to the query in an order corresponding to the timestamps.

22. The apparatus as recited in claim 14, further comprising:

a subsystem, implemented at least partially in hardware, that receives one or more event identifiers, each event identifier corresponding to an event in the events responsive to the query;

a subsystem, implemented at least partially in hardware, that retrieves from the distributed machines further information about the event corresponding to the event identifier using one of the event identifiers.

23. The apparatus as recited in claim 14, further comprising:

a subsystem, implemented at least partially in hardware, that displays a histogram of the events responsive to the query, wherein the histogram indicates how many of the events responsive to the query have a timestamp falling within each of a plurality of time ranges.

24. The apparatus as recited in claim 14, wherein the criterion for searching the set of events includes a time range to be compared to the timestamps of the events.

25. The apparatus as recited in claim 14, wherein the criterion for searching the set of events includes a value, set of values, or range of values for a field.

26. The apparatus as recited in claim 14, wherein the criterion for searching the set of events includes a specified keyword.

27. A non-transitory computer-readable medium storing one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform:

receiving a query including a criterion for searching a set of events stored across a plurality of distributed machines, wherein each distributed machine has responsibility to search a time-based subgroup of the set of events, wherein each distributed machine has responsibility to search a time-based subgroup different than time-based subgroups that other distributed machines have responsibility to search, and wherein each event in the set of events is associated with a timestamp;

in response to receiving the query, directing at least a portion of the plurality of distributed machines to search, in respective time-based subgroups, for events responsive to the query;

receiving from the distributed machines information relating to values for a field that are in the events responsive to the query;

aggregating the received information to determine a number corresponding to how many unique values exist for the field in the events responsive to the query;

displaying the number corresponding to how many unique values exist for the field.

28. The non-transitory computer-readable medium as recited in claim 27, wherein the events include machine data.

29. The non-transitory computer-readable medium as recited in claim 27, wherein the events include log data.

30. The non-transitory computer-readable medium as recited in claim 27, wherein the events include unstructured data.

31. The non-transitory computer-readable medium as recited in claim 27, wherein the criterion is an evaluation of values for the field.

32. The non-transitory computer-readable medium as recited in claim 27, further comprising:

for each event in at least a portion of the events responsive to the query, displaying its value for the field.

33. The non-transitory computer-readable medium as recited in claim 27, further comprising:

receiving from the distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query; and

displaying at least a portion of the information about the events responsive to the query.

34. The non-transitory computer-readable medium as recited in claim 27, further comprising:

receiving from the at least a portion of the plurality of distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query;

receiving timestamps associated with the events to which the information pertains;

displaying at least a portion of the information about the events responsive to the query in an order corresponding to the timestamps.

35. The non-transitory computer-readable medium as recited in claim 27, further comprising:

receiving one or more event identifiers, each event identifier corresponding to an event in the events responsive to the query;

retrieving from the distributed machines further information about the event corresponding to the event identifier using one of the event identifiers.

36. The non-transitory computer-readable medium as recited in claim 27, further comprising:

displaying a histogram of the events responsive to the query, wherein the histogram indicates how many of the events responsive to the query have a timestamp falling within each of a plurality of time ranges.

37. The non-transitory computer-readable medium as recited in claim 27, wherein the criterion for searching the set of events includes a time range to be compared to the timestamps of the events.

38. The non-transitory computer-readable medium as recited in claim 27, wherein the criterion for searching the set of events includes a value, set of values, or range of values for a field.

39. The non-transitory computer-readable medium as recited in claim 27, wherein the criterion for searching the set of events includes a specified keyword.