Real time searching and reporting

• US 8,412,696 B2
• Filed: 01/31/2011
• Issued: 04/02/2013
• Est. Priority Date: 01/31/2011
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Zhang, Steve Y., Sorkin, Stephen P., Patel, Vishal
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US13/018,236
• Publication Number:
  US 20120197928A1
• Time in Patent Office: 792 Days
• Field of Search: 707/2, 707/3, 707/706, 709/225, 700/108
• US Class Current: 707/706
• CPC Class Codes: G06F 16/2471 Distributed queries

Abstract

A system arranged to search machine data to generate reports in real time. A search query is provided that includes a plurality of search commands. The search query is parsed to form a main search query and a remote search query. Machine data is collected from remote data sources and evaluated against one of the main and remote search queries to generate a set of search results. The main search query is then evaluated against at least a partial set of the search result to generate at least one report regarding the collected machine data. Each report can be provided for display to a user.

Claims

1. A method for searching and reporting machine data with a computing device over a network, comprising:

receiving a search query comprising a plurality of commands;

employing a plurality of commands in the search query to generate a remote search query and a main search query;

receiving machine data from at least one of a plurality of remote data sources that is then aggregated into a plurality of separate events, wherein each event includes at least one determined segment;

performing the remote search query against the collected machine data to generate a search result associated with the remote search query; and

performing the main search query against the search result to generate a report of the collected machine data.

2. The method of claim 1, wherein the plurality of commands includes a pipelined sequence of search commands.

3. The method of claim 1, wherein the search query is parsed to form the remote search query and the main search query based on whether commands in the plurality of commands are distributable.

4. The method of claim 1, wherein receiving machine data comprises collecting the machine data after receiving the search query.

5. The method of claim 1, wherein receiving the machine data comprises dividing the machine data into events, and performing the remote search query against the collected machine data comprises performing the remote search query against the events.

6. The method of claim 1, wherein the machine data includes at least one of a server log, an activity log and a database record.

7. The method of claim 1, wherein the remote search query is performed responsive to a determination that the search query is a real-time search query.

8. The method of claim 1, further comprising:

receiving additional machine data;

performing the remote search query against the additional machine data to generate an updated search result; and

performing the main search query against the updated search result to generate an updated report.

9. The method of claim 1, further comprising outputting each report for display.

10. The method of claim 1, further comprising:

storing the collected machine data into a data store;

receiving another search query; and

if the other search query is determined to be non-real time, performing he other search query on the machine data stored in the data store.

11. A non-transitory computer-readable storage medium that comprises computer program code for searching and reporting machine data over a network, wherein execution of the computer program code by a processor enables actions, including:

receiving a search query comprising a plurality of commands;

employing a plurality of commands in the search query to generate a remote search query and a main search query;

receiving machine data from at least one of a plurality of remote data sources that is then aggregated into a plurality of separate events, wherein each event includes at least one determined segment;

performing the remote search query against the collected machine data to generate a search result associated with the remote search query; and

performing the main search query against the search result to generate a report of the collected machine data.

12. The medium of claim 11, wherein the plurality of commands includes a pipelined sequence of search commands.

13. The medium of claim 11, wherein the search query is parsed to form the remote search query and the main search query based on whether commands in the plurality of commands are distributable,

14. The medium of claim 11, wherein collecting machine data comprises collecting the machine data after receiving the search query.

15. The medium of claim 11, wherein the remote search query is performed responsive to a determination that the search query is real-time.

16. The medium of claim 11, wherein the actions further comprise:

receiving additional machine data;

performing the remote search query against the additional machine data to generate an updated search result; and

performing the main search query against the updated search result to generate an updated report.

17. A network device for searching and reporting machine data over a network, comprising:

a transceiver for communicating over a network;

a memory for storing, a plurality of data; and

a processor for executing the data to perform actions, including;

receiving a search query comprising a plurality of commands;

employing a plurality of commands in the search query to generate a remote search query and a main search query;

receiving machine data from at least one of a plurality of remote data sources that is then aggregated into a plurality of separate events, wherein each event includes at least one determined segment;

performing the remote search query against the collected machine data to generate a search result associated with the remote search query; and

performing the main search query against the search result to generate a report of the collected machine data,

18. The network device of claim 17, wherein the plurality of commands includes a pipelined sequence of search commands.

19. The network device of claim 17, wherein the search query is parsed to form the remote search query and the main search query based on whether commands in the plurality of commands are distributable.

20. The network device of claim 17, wherein receiving machine data comprises collecting the machine data after receiving the search query.

21. The network device of claim 17, wherein receiving the machine data comprises dividing the machine data into events, and performing the remote search query against the collected machine data comprises performing the remote search query against the events.

22. The network device of claim 17, wherein the machine data includes at least one of a server log, an activity log and a database record.

23. The network device of claim 17, wherein the remote search query is performed responsive to a determination that the search query is a. real-time search query.

24. The network device of claim 17, further comprising:

receiving additional machine data;

performing the remote search query against the additional machine data to generate an updated search result; and

performing the main search. query against the updated search result to generate an updated report.

25. The network device of claim 17, further comprising outputting each report for display.

26. The network device of claim 17, further comprising:

storing the collected machine data into a data store;

receiving another search query; and

if the other search query is determined to be non-real time, performing the other search query on the machine data stored in the data store.

27. A system for searching and reporting machine data over a network, comprising:

a server device that performs actions;

including;

receiving a search query comprising a plurality of commands;

employing a plurality of commands in the search query to generate a remote search query and a main search query;

receiving machine data from at least one of a plurality of remote data sources that is then aggregated into a plurality of separate events, wherein each event includes at least one determined segment;

performing the remote search query against the collected machine data to generate a search result associated with the remote search query; and

performing the main search query against the search result to generate a report of the collected machine data; and

a client device that performs actions including;

providing the search query to the server device; and

displaying the report to a user.

28. The network device of claim 27, wherein the search query is parsed to form the remote search query and the main search query based on whether commands in the plurality of commands are distributable.

29. The network device of claim 27, wherein receiving the machine data comprises dividing the machine data into events, and performing the remote search query against the collected machine data comprises performing the remote search query against the events.

30. The network device of claim 27, further comprising:

receiving additional machine data;

performing the remote search query against the additional machine data to generate an updated search result; and

performing the main search query against the updated search result to generate an updated report for display by the client device.

31. The network device of claim 27, farther comprising:

storing the collected machine data into a data store;

receiving another search query; and

if the other search query is determined to be non-real time, performing the other search query on the machine data stored in the data store.