Dynamic search guidance for machine data indexing and search system

• US 10,679,142 B2
• Filed: 04/29/2016
• Issued: 06/09/2020
• Est. Priority Date: 04/29/2016
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Ganapathi, Archana Sulochana
• Primary Examiner(s): Nilsson, Eric
• Assistant Examiner(s): Kim, Sehwan
• Application Number:
  US15/142,346
• Publication Number:
  US 20170316337A1
• Time in Patent Office: 1,502 Days
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/047 Pattern matching networks; Rete networks

Abstract

Disclosed is a guidance technique that can be applied to guide search and analysis of stored data by a user. The technique can include inputting from a user a portion of a search query expressed in a pipelined search language, at a system for indexing and searching machine data. The system generates and outputs search guidance for the user as the user builds the search query, by applying the portion of the query to an operation flow model, where the operation flow model represents a plurality of searches performable by the system. The operation flow model has been generated based on multi-user historical search data and includes a plurality of states, each representing a different group of related commands of the pipelined search language.

Claims

1. A method comprising:

inputting from a user a portion of a search query expressed in a pipelined search language, at a system for indexing and searching machine data;

generating, by the system for indexing and searching machine data, search guidance for the user prior to completion of the search query by the user, by applying the portion of the query to an operation flow model, the operation flow model including a plurality of states and representing a plurality of searches performable by the system for indexing and searching machine data, each state in the operation flow model representing a different group of related commands of the pipelined search language, the operation flow model having been generated based on multi-user historical search data, wherein generating the search guidance for the user includes determining a most probable next state of the search query based on the operation flow model as the user builds the search query and based on a field-value pair in a result of executing the portion of the search query, and wherein generating the search guidance for the user further includes

determining, by the system for indexing and searching machine data, that the result of executing the portion of the search query includes a particular field of a field-value pair, and

in response to the result of executing the portion of the search query including the particular field of the field-value pair, recommending, by the system for indexing and searching machine data, to the user a command that makes use of the particular field-value pair, as a next command to include in the search query; and

causing, by the system for indexing and searching machine data, the search guidance to be output to the user prior to completion of the search query by the user.

2. The method of claim 1, wherein the portion of the search query comprises a command and an argument of the command.

3. The method of claim 1, further comprising:

selecting the operation flow model from among a plurality of stored operation flow models, in response to the inputting of the portion of the search query.

4. The method of claim 1, further comprising:

selecting the operation flow model from among a plurality of stored operation flow models, in response to the inputting of the portion of the search query, wherein the search query specifies a search of a data source of machine data, and the operation flow model is specific to the data source.

5. The method of claim 1, further comprising:

identifying a data source specified by the search query; and

selecting the operation flow model from among a plurality of stored operation flow models, based on the data source specified by the search query, each of the plurality of stored operation flow models corresponding to a different one of a plurality of data source types.

6. The method of claim 1, wherein:

the multi-user historical search data comprises past search queries submitted by a plurality of users on one or more instances of the system for indexing and searching machine data;

the operation flow model includes a plurality of transitions, each transition representing a relationship between two of the plurality of states; and

at least some of the transitions are weighted according to their frequencies of occurrence in the multi-user historical search data.

7. The method of claim 1, wherein the operation flow model includes a plurality of transitions, each transition representing a relationship between two of the plurality of states, and wherein at least some of the transitions are weighted according to their frequencies of occurrence in the multi-user historical search data.

8. The method of claim 1, wherein each state in the operation flow model represents a group of related but independently executable pipelined search language commands of the system for indexing and searching machine data.

9. The method of claim 1, wherein:

each state in the operation flow model represents a group of related but independently executable pipelined search language commands of the system for indexing and searching machine data; and

the plurality of states are defined such that at least one of the following is true;

all of the pipelined search language commands of each state are of the same or similar type;

all of the pipelined search language commands of each state have the same or similar types of arguments;

or

all of the pipelined search language commands of each state have the same or similar numbers of arguments.

10. The method of claim 1, wherein said generating search guidance for the user comprises determining a most probable next state of the search query based on the operation flow model, as the user builds the search query.

11. The method of claim 1, wherein:

said generating search guidance for the user comprises determining a most probable next state of the search query based on the operation flow model, as the user builds the search query;

the most probable next state of the search query represents a subset of all search commands of the pipelined search language; and

said generating search guidance for the user further comprises causing an indication of the commands of said subset to be output to the user as the user builds the search query.

12. The method of claim 1, wherein:

said operation flow model is one of a plurality of stored operation flow models, each of the operation flow models representing a different plurality of searches performable by the system for indexing and searching machine data; and

said generating search guidance for the user is based on the plurality of operation flow models.

13. The method of claim 1, wherein:

said operation flow model is one of a plurality of stored operation flow models, each of the operation flow models representing a different plurality of searches performable by the system for indexing and searching machine data; and

said generating search guidance for the user comprises voting across the plurality of operation flow models to determine the search guidance.

14. The method of claim 1, wherein said generating search guidance for the user comprises determining a plurality of most probable next states of the search query based on the operation flow model, as the user builds the search query, each of the determined most probable next states representing a different subset of all commands of the pipelined search language.

15. The method of claim 1, wherein:

said generating search guidance for the user comprises determining a plurality of most probable next states of the search query based on the operation flow model, as the user builds the search query, each of the determined most probable next states representing a different subset of all commands of the pipelined search language;

said generating search guidance for the user further comprises causing an indication of the plurality of most probable next states to be output to the user as the user builds the search query;

the method further comprising;

receiving user input directed to one of the most probable next states in the indication; and

in response to the user input, causing output to the user of an indication of the pipelined search language commands represented by said one of the most probable next states.

16. The method of claim 1, further comprising:

generating the operation flow model by executing a machine-learning algorithm.

17. The method of claim 1, further comprising:

generating a plurality of operation flow models, including said operation flow model, each of the plurality of operation flow models representing a different plurality of searches performable by the system for indexing and searching machine data, based on the multi-user historical search data.

18. The method of claim 1, further comprising:

generating a plurality of operation flow models, including said operation flow model, each of the plurality of operation flow models representing a different plurality of searches performable by the system for indexing and searching machine data, based on the multi-user historical search data;

wherein said generating a plurality of operation flow models comprises applying a machine learning algorithm to the multiuser historical search data.

19. The method of claim 1, further comprising:

defining, by a computer system, the plurality of states of the operation flow model based on stored signature data of a plurality of pipelined search language commands of the system for indexing and searching machine data.

20. The method of claim 1, further comprising:

defining, by a computer system, the plurality of states of the operation flow model based on stored signature data of a plurality of pipelined search language commands of the system for indexing and searching machine data;

wherein defining the plurality of states is performed by applying a machine learning algorithm to the signature data.

21. The method of claim 1, further comprising:

defining, by a computer system, the plurality of states of the operation flow model based on stored signature data of a plurality of pipelined search language commands of the system for indexing and searching machine data; and

generating the operation flow model by executing a machine-learning algorithm.

22. The method of claim 1, further comprising:

defining the plurality of states of the operation flow model by applying a first machine learning algorithm to stored signature data of a plurality of pipelined search language commands of the system for indexing and searching machine data; and

generating the operation flow model by applying a second machine-learning algorithm to the plurality of states and the multi-user historical search data.

23. The method of claim 1, further comprising:

defining states of a plurality of operation flow models by applying a first machine learning algorithm to stored signature data of a plurality of pipelined search language commands of the system for indexing and searching machine data, each of the plurality of operation flow models representing a different plurality of possible searches by the system for indexing and searching machine data; and

generating the plurality of operation flow models by applying a second machine-learning algorithm to the defined states and the multi-user historical search data.

24. A system for indexing and searching machine data, the system comprising:

a communication device through which to communicate on a computer network; and

at least one processor operatively coupled to the communication device and configured to execute operations including

inputting from a user a portion of a search query in a pipelined search language of the system for indexing and searching machine data;

generating search guidance for the user prior to completion of the search query by the user, by applying the portion of the query to an operation flow model, the operation flow model including a plurality of states and representing a plurality of searches performable by the system for indexing and searching machine data, each state representing a different group of related commands of the pipelined search language, the operation flow model having been generated based on multi-user historical search data, wherein generating the search guidance for the user includes determining a most probable next state of the search query based on the operation flow model as the user builds the search query and based on a field-value pair in a result of executing the portion of the search query, and wherein generating the search guidance for the user further includes

determining, by the system for indexing and searching machine data, that the result of executing the portion of the search query includes a particular field of a field-value pair, and

in response to the result of executing the portion of the search query including the particular field of the field-value pair, recommending, by the system for indexing and searching machine data, to the user a command that makes use of the particular field-value pair, as a next command to include in the search query; and

causing the search guidance to be output to the user prior to completion of the search query by the user.

25. A non-transitory machine-readable storage medium for use in a processing system, the non-transitory machine-readable storage medium storing instructions, an execution of which in the processing system causes the processing system to perform operations comprising:

inputting from a user a portion of a search query expressed in a pipelined search language of a system for indexing and searching machine data;

generating search guidance for the user prior to completion of the search query by the user, by applying the portion of the query to an operation flow model, the operation flow model including a plurality of states and representing a plurality of searches performable by the system for indexing and searching machine data, each state representing a different group of related commands of the pipelined search language, the operation flow model having been generated based on multi-user historical search data, wherein generating the search guidance for the user includes determining a most probable next state of the search query based on the operation flow model as the user builds the search query and based on a field-value pair in a result of executing the portion of the search query, and wherein generating the search guidance for the user further includes

determining, by the system for indexing and searching machine data, that the result of executing the portion of the search query includes a particular field of a field-value pair, and

in response to the result of executing the portion of the search query including the particular field of the field-value pair, recommending, by the system for indexing and searching machine data, to the user a command that makes use of the particular field-value pair, as a next command to include in the search query; and

causing the search guidance to be output to the user prior to completion of the search query by the user.