Discovering fields to filter data returned in response to a search

• US 9,582,585 B2
• Filed: 07/31/2014
• Issued: 02/28/2017
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Neels, Alice Emily, Vasan, Sundar, Fishel, Simon, Robichaud, Marc Vincent
• Primary Examiner(s): Casanova, Jorge A
• Application Number:
  US14/448,937
• Publication Number:
  US 20150026167A1
• Time in Patent Office: 943 Days
• Field of Search: 707/723
• US Class Current: 1/1
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

Fields may be discovered in events that are returned in response to an initial search. The events may comprise portions of raw data. Furthermore, the fields may be defined by extraction rules for extracting values from corresponding portions of raw data. The displaying of a graphical user interface (GUI) may be caused where the GUI enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar. At least one criterion for at least one field from the subset of the discovered fields may be received through a portion of the GUI that does not include a search bar for entering a search query. The events returned in response to the initial search query may be caused to be filtered based on the received criterion.

Claims

1. A method comprising:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

determining a number of the returned events that comprise a first field of the discovered fields; and

calculating a score for the first field based on the number of the returned events that comprise the first field, wherein a first field name associated with the first field is selected to be displayed in the GUI based on the calculated score.

2. The method of claim 1, wherein the raw data included in the events includes machine data.

3. The method of claim 1, wherein the fields are pre-defined fields.

4. The method of claim 1, wherein the fields are discovered as the events are accessed.

5. The method of claim 1, further comprising:

causing display of a visualization of data in the filtered events.

6. A method comprising:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

determining a number of unique values of a first field of the discovered fields; and

calculating a score for the first field based on the number of unique values of the first field, wherein a first field name associated with the first field is selected to be displayed in the GUI based on the calculated score.

7. The method of claim 6, wherein the raw data included in the events includes machine data.

8. The method of claim 6, wherein the fields are pre-defined fields.

9. The method of claim 6, wherein the fields are discovered as the events are accessed.

10. The method of claim 6, further comprising:

causing display of a visualization of data in the filtered events.

11. A method comprising:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

calculating a score for each of the discovered fields; and

receiving a threshold percentage of the discovered fields to display in the GUI, wherein the set of field names associated with at least a portion of the discovered fields are selected to be displayed in the GUI based on the threshold percentage and the calculated score for each of the discovered fields, wherein the threshold percentage indicates a percentage of the discovered fields to be selected to be displayed in the GUI.

12. The method of claim 11, wherein the raw data included in the events includes machine data.

13. The method of claim 11, wherein the fields are pre-defined fields.

14. The method of claim 11, wherein the fields are discovered as the events are accessed.

15. The method of claim 11, further comprising:

causing display of a visualization of data in the filtered events.

16. A method comprising:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

generating a data model based on discovered fields and the initial search query.

17. The method of claim 16, wherein the raw data included in the events includes machine data.

18. The method of claim 16, wherein the fields are pre-defined fields.

19. The method of claim 16, wherein the fields are discovered as the events are accessed.

20. The method of claim 16, further comprising:

causing display of a visualization of data in the filtered events.

21. A method comprising:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

generating a data model based on the discovered fields; and

modifying a search defining events to which a data model is applicable based on the one or more discovered fields associated with the selected one or more field names.

22. The method of claim 21, wherein the raw data included in the events includes machine data.

23. The method of claim 21, wherein the fields are pre-defined fields.

24. The method of claim 21, wherein the fields are discovered as the events are accessed.

25. The method of claim 21, further comprising:

causing display of a visualization of data in the filtered events.

26. A method comprising:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

generating a data model based on the initial search query and the discovered fields;

saving the data model; and

applying the data model to a different data set than was searched using the initial search query.

27. The method of claim 26, wherein the raw data included in the events includes machine data.

28. The method of claim 26, wherein the fields are pre-defined fields.

29. The method of claim 26, wherein the fields are discovered as the events are accessed.

30. The method of claim 26, further comprising:

causing display of a visualization of data in the filtered events.

31. A system comprising:

a memory; and

a processing device coupled to the memory, to;

access events returned in response to an initial search query, the accessed events comprising portions of raw data;

discover fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

cause display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receive through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

cause the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

determine a number of the returned events that comprise a first field of the discovered fields; and

calculate a score for the first field based on the number of the returned events that comprise the first field, wherein a first field name associated with the first field is selected to be displayed in the GUI based on the calculated score.

32. The system of claim 31, wherein the raw data included in the events includes machine data.

33. A system comprising:

a memory; and

a processing device coupled to the memory, to;

access events returned in response to an initial search query, the accessed events comprising portions of raw data;

discover fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

cause display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receive through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

cause the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

determine a number of unique values of a first field of the discovered fields; and

calculate a score for the first field based on the number of unique values of the first field, wherein a first field name associated with the first field is selected to be displayed in the GUI based on the calculated score.

34. The system of claim 33, wherein the raw data included in the events includes machine data.

35. A system comprising:

a memory; and

a processing device coupled to the memory, to;

access events returned in response to an initial search query, the accessed events comprising portions of raw data;

discover fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

cause display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receive through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

cause the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

generate a data model based on discovered fields and the initial search query.

36. The system of claim 35, wherein the raw data included in the events includes machine data.

37. One or more non-transitory computer readable storage media storing instructions which, when executed by one or more computing devices, cause:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field;

determining a number of the returned events that comprise a first field of the discovered fields; and

calculating a score for the first field based on the number of the returned events that comprise the first field, wherein a first field name associated with the first field is selected to be displayed in the GUI based on the calculated score.

38. The one or more non-transitory storage media of claim 37, wherein the raw data included in the events includes machine data.

39. One or more non-transitory computer readable storage media storing instructions which, when executed by one or more computing devices, cause:

accessing events returned in response to an initial search query, the accessed events comprising portions of raw data;

discovering fields in the accessed events, the fields being defined by extraction rules each comprising a regular expression for extracting values from corresponding portions of raw data;

causing display of a graphical user interface (GUI) that enables a user to select or enter criteria for a subset of the discovered fields without entering a search query in a search bar;

receiving through a portion of the GUI that does not include a search bar for entering a search query at least one criterion for at least one field from the subset of the discovered fields;

causing, by a processing device, the events returned in response to the initial search query to be filtered based on the received at least one criterion for the at least one field; and

generating a data model based on discovered fields and the initial search query.

40. The one or more non-transitory storage media of claim 39, wherein the raw data included in the events includes machine data.