Scalable Interactive Display Of Distributed Data

• US 20140317111A1
• Filed: 05/01/2014
• Published: 10/23/2014
• Est. Priority Date: 03/14/2011
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Zhang, Steve Yu, Sorkin, Stephen Phillip
• Granted Patent: US 9,245,039 B2
• Time in Patent Office: Days
• US Class Current: 707/736
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/90328 using search space presentation or visualization, e.g. category or range presentation and selection
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04L 41/0604 using filtering, e.g. reduction of information by using priority, element types, position or time
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a “divide and conquer” algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.

Claims

1. A method, comprising:

receiving, at a first device, a search query to be performed on a set of event records accessible by the first device;

searching, by the first device, the set of event records using the search query;

sending, by the first device, a search result to a second device, the search result including event identifiers that are associated with event records of the set of event records that satisfied the search query;

after sending the search result to the second device, receiving, by the first device from the second device, at least one event identifier that was included in the search result;

based on receiving the at least one event identifier from the second device, sending, by the first device, at least one event record associated with the received at least one event identifier to the second device.

2. The method as recited in claim 1, wherein each event record in the set of event records is associated with a time stamp.

3. The method as recited in claim 1, wherein each event record in the set of event records is associated with a time stamp and is searchable based on a time represented by the time stamp.

4. The method as recited in claim 1, wherein an event identifier includes an identification of both an event record and a device having access to the event record.

5. The method as recited in claim 1, wherein the received at least one event identifier received by the first device includes an identification of both an event record and the first device.

6. The method as recited in claim 1, wherein the search result sent by the first device to the second device does not include an event record associated with an event identifier that was included in the search result.

7. The method as recited in claim 1, wherein the first device includes an indexer.

8. The method as recited in claim 1, wherein the second device includes a search head.

9. The method as recited in claim 1, wherein the first device includes an indexer, and wherein the second device includes a search head.

10. An apparatus, comprising:

a subsystem, at a first device, implemented at least partially in hardware, that receives a search query to be performed on a set of event records accessible by the first device;

a subsystem, at a first device, implemented at least partially in hardware, that searches the set of event records using the search query;

a subsystem, at a first device, implemented at least partially in hardware, that sends a search result to a second device, the search result including event identifiers that are associated with event records of the set of event records that satisfied the search query;

a subsystem, at a first device, implemented at least partially in hardware, that after sending the search result to the second device, receives from the second device, at least one event identifier that was included in the search result;

a subsystem, at a first device, implemented at least partially in hardware, that based on receiving the at least one event identifier from the second device, sends at least one event record associated with the received at least one event identifier to the second device.

11. The apparatus as recited in claim 10, wherein each event record in the set of event records is associated with a time stamp.

12. The apparatus as recited in claim 10, wherein each event record in the set of event records is associated with a time stamp and is searchable based on a time represented by the time stamp.

13. The apparatus as recited in claim 10, wherein an event identifier includes an identification of both an event record and a device having access to the event record.

14. The apparatus as recited in claim 10, wherein the received at least one event identifier received by the first device includes an identification of both an event record and the first device.

15. The apparatus as recited in claim 10, wherein the search result sent by the first device to the second device does not include an event record associated with an event identifier that was included in the search result.

16. The apparatus as recited in claim 10, wherein the first device includes an indexer.

17. The apparatus as recited in claim 10, wherein the second device includes a search head.

18. The apparatus as recited in claim 10, wherein the first device includes an indexer, and wherein the second device includes a search head.

19. A non-transitory computer-readable medium storing one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform:

receiving, at a first device, a search query to be performed on a set of event records accessible by the first device;

searching, by the first device, the set of event records using the search query;

sending, by the first device, a search result to a second device, the search result including event identifiers that are associated with event records of the set of event records that satisfied the search query;

after sending the search result to the second device, receiving, by the first device from the second device, at least one event identifier that was included in the search result;

based on receiving the at least one event identifier from the second device, sending, by the first device, at least one event record associated with the received at least one event identifier to the second device.

20. The non-transitory computer-readable medium as recited in claim 19, wherein each event record in the set of event records is associated with a time stamp.

21. The non-transitory computer-readable medium as recited in claim 19, wherein each event record in the set of event records is associated with a time stamp and is searchable based on a time represented by the time stamp.

22. The non-transitory computer-readable medium as recited in claim 19, wherein an event identifier includes an identification of both an event record and a device having access to the event record.

23. The non-transitory computer-readable medium as recited in claim 19, wherein the received at least one event identifier received by the first device includes an identification of both an event record and the first device.

24. The non-transitory computer-readable medium as recited in claim 19, wherein the search result sent by the first device to the second device does not include an event record associated with an event identifier that was included in the search result.

25. The non-transitory computer-readable medium as recited in claim 19, wherein the first device includes an indexer.

26. The non-transitory computer-readable medium as recited in claim 19, wherein the second device includes a search head.

27. The non-transitory computer-readable medium as recited in claim 19, wherein the first device includes an indexer, and wherein the second device includes a search head.