Scalable interactive display of distributed data

• US 8,751,529 B2
• Filed: 10/25/2012
• Issued: 06/10/2014
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Zhang, Steve Yu, Sorkin, Stephen Phillip
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Liu, Hexing
• Application Number:
  US13/660,845
• Publication Number:
  US 20130073542A1
• Time in Patent Office: 593 Days
• Field of Search: None
• US Class Current: 707/770
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/90328 using search space presentation or visualization, e.g. category or range presentation and selection
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a “divide and conquer” algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

Claims

1. A computer-implemented method, comprising:

maintaining a set of time stamped events in an indexed data store, wherein the indexed data store is accessible by one or more computer servers in a plurality of computer servers;

sending a request to query the time stamped events, wherein the query is executable by a distributed server in the plurality of computer servers, and wherein the distributed server is responsible for searching a portion of the set of time stamped events;

receiving result information including one or more event identifiers, wherein the result information is based on one or more event records that satisfy the query, wherein each event identifier corresponds to an event record, and wherein an event identifier includes information for locating the event record;

aggregating the result information that satisfies the query;

displaying the aggregated result information;

receiving input requesting additional information based on one or more time stamped events that satisfy the request;

determining one or more event records that include the additional information;

determining an event identifier that corresponds to each of the determined one or more event records;

sending a new request for the determined one or more event records, wherein event identifiers are used to locate corresponding event records accessible on the distributed server;

receiving the determined one or more event records that satisfy the new request; and

displaying the requested additional information.

2. The method of claim 1, wherein the event record includes event information associated with a time stamped event that satisfies the request.

3. The method of claim 1, wherein the event record includes a time stamped event that is relevant to the request.

4. The method of claim 1, wherein the event identifier indicates a computer server that stores an event record and a value indicating a location of the event record in the computer server.

5. The method of claim 1, wherein the plurality of computer servers execute the request to query the time stamped events concurrently.

6. The method of claim 1, wherein the result information indicates statistical information about one or more time stamped events that satisfy the request.

7. The method of claim 1, wherein the query identifies the plurality of computer servers that execute the query.

8. The method of claim 1, further comprising:

periodically querying the time stamped events; and

periodically generating a report.

9. A computer-implemented system, comprising:

one or more processors; and

one or more non-transitory computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including;

maintaining a set of time stamped events in an indexed data store, wherein the indexed data store is accessible by one or more computer servers in a plurality of computer servers;

sending a request to query the time stamped events, wherein the query is executable by a distributed server in the plurality of computer servers, and wherein the distributed server is responsible for searching a portion of the set of time stamped events;

receiving result information including one or more event identifiers, wherein the result information is based on one or more event records that satisfy the query, wherein each event identifier corresponds to an event record, and wherein an event identifier includes information for locating the event record;

aggregating the result information that satisfies the query;

displaying the aggregated result information;

receiving input requesting additional information based on one or more time stamped events that satisfy the request;

determining one or more event records that include the additional information;

determining an event identifier that corresponds to each of the determined one or more event records;

sending a new request for the determined one or more event records, wherein event identifiers are used to locate corresponding event records accessible on the distributed server;

receiving the determined one or more event records that satisfy the new request; and

displaying the requested additional information.

10. The system of claim 9, wherein the event record includes event information associated with a time stamped event that satisfies the request.

11. The system of claim 9, wherein the event record includes a time stamped event that is relevant to the request.

12. The system of claim 9, wherein the event identifier indicates a computer server that stores an event record and a value indicating a location of the event record in the computer server.

13. The system of claim 9, wherein the plurality of computer servers execute the request to query the time stamped events concurrently.

14. The system of claim 9, wherein the result information indicates statistical information about one or more time stamped events that satisfy the request.

15. The system of claim 9, wherein the query identifies the plurality of computer servers that execute the query.

16. The system of claim 9, wherein the one or more non-transitory computer-readable storage mediums contain further instructions configured to cause the one or more processors to perform operations including:

periodically querying the time stamped events; and

periodically generating a report.

17. A computer-program product, tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to:

maintain a set of time stamped events in an indexed data store, wherein the indexed data store is accessible by one or more computer servers in a plurality of computer servers;

send a request to query the time stamped events, wherein the query is executable by a distributed server in the plurality of computer servers, and wherein the distributed server is responsible for searching a portion of the set of time stamped events;

receive result information including one or more event identifiers, wherein the result information is based on one or more event records that satisfy the query, wherein each event identifier corresponds to an event record, and wherein an event identifier includes information for locating the event record;

aggregate the result information that satisfies the query;

display the aggregated result information;

receive input requesting additional information based on one or more time stamped events that satisfy the request;

determine one or more event records that include the additional information;

determine an event identifier that corresponds to each of the determined one or more event records;

send a new request for the determined one or more event records, wherein event identifiers are used to locate corresponding event records accessible on the distributed server;

receive the determined one or more event records that satisfy the new request; and

display the requested additional information.

18. The computer-program product of claim 17, wherein the event record includes a time stamped event that is relevant to the request.

19. The computer-program product of claim 17, wherein the event identifier indicates a computer server that stores an event record and a value indicating a location of the event record in the computer server.

20. The computer-program product of claim 17, wherein the result information indicates statistical information about one or more time stamped events that satisfy the request.