Determination and display of the number of unique values for a field defined for events in a distributed data store

• US 8,990,245 B2
• Filed: 01/17/2014
• Issued: 03/24/2015
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Zhang, Steve Yu, Sorkin, Stephen Phillip
• Primary Examiner(s): Beausoliel, Robert W.
• Assistant Examiner(s): Liu, Hexing
• Application Number:
  US14/158,421
• Publication Number:
  US 20140136529A1
• Time in Patent Office: 431 Days
• Field of Search: None
• US Class Current: 707/770
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/90328 using search space presentation or visualization, e.g. category or range presentation and selection
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a “divide and conquer” algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

Claims

1. A computer-implemented method, comprising:

receiving a query including a criterion for searching a set of events stored across a plurality of distributed machines, wherein each distributed machine has access to search a subgroup of the stored set of events, wherein each distributed machine has access to search a subgroup different than subgroups that other distributed machines have access to search, and wherein each event is associated with a timestamp;

in response to receiving the query, directing the plurality of distributed machines to search, in respective subgroups to which they have access, for events responsive to the query;

receiving from the distributed machines information about values for a field that are extracted from the events responsive to the query;

synthesizing the information about the values for the field to determine a number corresponding to how many unique values exist for the field in the events responsive to the query;

displaying a field name representing the field and the number corresponding to how many unique values exist for the field in a field picker, wherein the field picker lists a plurality of field names corresponding to fields defined for the events responsive to the query;

displaying information about a subset of the events that are both responsive to the query and that meet a criterion for a field corresponding to a field name selected from the list of the plurality of field names.

2. The method of claim 1, wherein the events include machine data.

3. The method of claim 1, wherein the events include log data.

4. The method of claim 1, wherein the events include unstructured data.

5. The method of claim 1, wherein a second field name is displayed in the field picker and represents a second field defined for the events responsive to the query, and wherein a number corresponding to how many unique values exist for the second field in the events responsive to the query is displayed in association with the second field name.

6. The method of claim 1, wherein the criterion is an evaluation of values for the field corresponding to the field name selected.

7. The method of claim 1, wherein the method further comprises:

for each event in the subset of the events responsive to the query, displaying its value for the field corresponding to the selected field name.

8. The method of claim 1, wherein the method further comprises:

receiving the criterion for the field corresponding to the selected field name.

9. The method of claim 1, further comprising:

receiving from the distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query; and

displaying the information about the events responsive to the query.

10. The method of claim 1, further comprising:

receiving from the distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query;

receiving the timestamps associated with the events to which the information pertains; and

displaying the information about the events responsive to the query in an order corresponding to the timestamps.

11. The method of claim 1, further comprising:

receiving one or more event identifiers, each event identifier corresponding to an event in the events responsive to the query; and

using one of the event identifiers to retrieve from the distributed machines further information about the event corresponding to the event identifier.

12. The method of claim 1, further comprising:

displaying a histogram of the events responsive to the query, wherein the histogram indicates how many of the events responsive to the query have a timestamp falling within each of a plurality of time ranges.

13. The method of claim 1, wherein the criterion for searching the set of events includes a time range to be compared to the timestamps of the events.

14. The method of claim 1, wherein the criterion for searching the set of events includes a value, set of values, or range of values for a field.

15. The method of claim 1, wherein the criterion for searching the set of events includes a specified keyword.

16. A computer-implemented system, comprising:

one or more processors; and

one or more non-transitory computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including;

receiving a query including a criterion for searching a the set of events stored across a plurality of distributed machines, wherein each distributed machine has access to search a subgroup of the stored set of events, wherein each distributed machine has access to search a subgroup different than subgroups that other distributed machines have access to search, and wherein each event is associated with a timestamp;

in response to receiving the query, directing the plurality of distributed machines to search, in respective subgroups to which they have access, for events responsive to the query;

receiving from the distributed machines information about values for a field extracted from the events responsive to the query;

synthesizing the information about the values for the field to determine a number corresponding to how many unique values exist for the field in the events responsive to the query;

displaying a field name representing the field and the number corresponding to how many unique values exist for the field in a field picker, wherein the field picker lists a plurality of field names corresponding to fields defined for the events responsive to the query;

displaying information about a subset of the events that are both responsive to the query and that meet a criterion for a field corresponding to a field name selected from the list of the plurality of field names.

17. The computer-implemented system of claim 16, wherein the criterion is an evaluation of values for the field corresponding to the field name selected.

18. The computer-implemented system of claim 16, the operations further comprising:

receiving from the distributed machines information about the events responsive to the query, wherein the information includes at least a portion of at least some of the events responsive to the query; and

displaying the information about the events responsive to the query.

19. A computer-program product, tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations, comprising:

receiving a query including a criterion for searching a set of events stored across a plurality of distributed machines, wherein each distributed machine has access to search a subgroup of the stored set of events, wherein each distributed machine has access to search a subgroup different than subgroups that other distributed machines have access to search, and wherein each event is associated with a timestamp;

in response to receiving the query, directing the plurality of distributed machines to search, in respective subgroups to which they have access, for events responsive to the query;

receiving from the distributed machines information about values for a field extracted from the events responsive to the query;

synthesizing the information about the values for the field to determine a number corresponding to how many unique values exist for the field in the events responsive to the query;

displaying a field name representing the field and the number corresponding to how many unique values exist for the field in a field picker, wherein the field picker lists a plurality of field names corresponding to fields defined for the events responsive to the query;

displaying information about a subset of the events that are both responsive to the query and that meet a criterion for a field corresponding to a field name selected from the list of the plurality of field names.

20. The computer-program product of claim 19, wherein the criterion is an evaluation of values for the field corresponding to the field name selected.