Wizard for configuring a field extraction rule

• US 10,783,324 B2
• Filed: 08/15/2019
• Issued: 09/22/2020
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Miller, Jesse, Delfino, Micah James, Robichaud, Marc, Hanson, Catherine Anne, Carasso, David
• Primary Examiner(s): Phantana-angkool, David
• Application Number:
  US16/541,637
• Publication Number:
  US 20200012715A1
• Time in Patent Office: 404 Days
• Field of Search: 715226
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/279 Recognition of textual entities

Abstract

The technology disclosed relates to formulating and refining field extraction rules that are used at query time on raw data with a late-binding schema. The field extraction rules identify portions of the raw data, as well as their data types and hierarchical relationships. These extraction rules are executed against very large data sets not organized into relational structures that have not been processed by standard extraction or transformation methods. By using sample events, a focus on primary and secondary example events help formulate either a single extraction rule spanning multiple data formats, or multiple rules directed to distinct formats. Selection tools mark up the example events to indicate positive examples for the extraction rules, and to identify negative examples to avoid mistaken value selection. The extraction rules can be saved for query-time use, and can be incorporated into a data model for sets and subsets of event data.

Claims

1. A computer-implemented method comprising:

providing a wizard configured to guide through configuring a field extraction rule to extract values for fields from events of raw data, wherein the wizard is configured to;

identify a user selection of a source type categorizing a data source of the events;

identify a user selection of a first example event of the events from the data source;

cause display of a field selection interface comprising a first area displaying the first example event with markups indicating (i) user selected tokens from text within the first example event and (ii) correspondence with the fields;

configure the field extraction rule to extract the user selected tokens from the first example event as the values for the fields; and

cause display, on a second area of the field selection interface, of a set of the events from the data source and preview extracted values for the fields, corresponding to the user selected tokens, resulting from applying the field extraction rule to the set of the events.

2. The method of claim 1, wherein the wizard is configured to provide a progress line indicating progress through a structured sequence from selecting the user selection of the source type, to selecting the user selection of the first example event, to selecting the user selected tokens, and concluding with saving the configured extraction rule.

3. The method of claim 1, wherein the wizard is configured to reduce a number of steps of the wizard by borrowing context from a state existing when the wizard is invoked.

4. The method of claim 1, wherein the wizard is configured to identify the user selection of the source type and the user selection of the first example event by recognizing that the source type and the first example event have been selected, while browsing with some other tool than the wizard, when the wizard is invoked.

5. The method of claim 1, wherein the wizard is configured to automatically configure the field extraction rule and provide a manual editor configured to accept and implement direct edits to the automatically configured field extraction rule.

6. The method of claim 1, wherein the preview extracted values for the fields comprise markups of text on the display of the set of the events from the data source.

7. The method of claim 1, wherein the preview extracted values for the fields are distinct from the display of the set of the events from the data source, and wherein the preview extracted values are arranged in columns.

8. The method of claim 1, wherein the wizard is configured to present the preview extracted values with an appearance based on a user selection between at least one of markups of text on the display of the set of the events or arranged in columns distinct from the display of the set of the events.

9. The method of claim 1, wherein the wizard is configured to receive a selection of a portion of text within a second example event, from the set of the events, and, based thereon, configure the field extraction rule as a more inclusive field extraction rule configured to extract both the portion of text from the second example event and the selected tokens from the first example event.

10. The method of claim 1, wherein the wizard is configured to accept a selection naming and saving the field extraction rule.

11. A system for configuring a field extraction rule, the system comprising:

one or more data processors; and

one or more computer-readable storage media containing instructions which when executed on the one or more data processors, cause the one or more processors to perform operations including;

providing a wizard configured to guide through configuring a field extraction rule to extract values for fields from events of raw data, wherein the wizard is configured to;

identify a user selection of a source type categorizing a data source of the events;

identify a user selection of a first example event of the events from the data source;

cause display of a field selection interface comprising a first area displaying the first example event with markups indicating (i) user selected tokens from text within the first example event and (ii) correspondence with the fields;

configure the field extraction rule to extract the user selected tokens from the first example event as the values for the fields; and

cause display, on a second area of the field selection interface, of a set of the events from the data source and preview extracted values for the fields, corresponding to the user selected tokens, resulting from applying the field extraction rule to the set of the events.

12. The system of claim 11, wherein the wizard is configured to provide a progress line indicating progress through a structured sequence from selecting the user selection of the source type, to selecting the user selection of the first example event, to selecting the user selected tokens, and concluding with saving the configured extraction rule.

13. The system of claim 11, wherein the wizard is configured to reduce a number of steps of the wizard by borrowing context from a state existing when the wizard is invoked.

14. The system of claim 11, wherein the wizard is configured to identify the user selection of the source type and the user selection of the first example event by recognizing that the source type and the first example event have been selected, while browsing with some other tool than the wizard, when the wizard is invoked.

15. The system of claim 11, wherein the wizard is configured to automatically configure the field extraction rule and provide a manual editor configured to accept and implement direct edits to the automatically configured field extraction rule.

16. The system of claim 11, wherein the preview extracted values for the fields comprise markups of text on the display of the set of the events from the data source.

17. The system of claim 11, wherein the preview extracted values for the fields are distinct from the display of the set of the events from the data source, and wherein the preview extracted values are arranged in columns.

18. The method of claim 11, wherein the wizard is configured to present the preview extracted values with an appearance based on a user selection between at least one of markups of text on the display of the set of the events or arranged in columns distinct from the display of the set of the events.

19. The system of claim 11, wherein the wizard is configured to receive a selection of a portion of text within a second example event, from the set of the events, and, based thereon, configure the field extraction rule as a more inclusive field extraction rule configured to extract both the portion of text from the second example event and the selected tokens from the first example event.

20. The system of claim 11, wherein the wizard is configured to accept a selection naming and saving the field extraction rule.

21. One or more computer-storage media storing computer-executable instructions that, when executed by a computing device, perform a method for configuring a field extraction rule, the method comprising:

providing a wizard configured to guide through configuring a field extraction rule to extract values for fields from events of raw data, wherein the wizard is configured to;

identify a user selection of a source type categorizing a data source of the events;

identify a user selection of a first example event of the events from the data source;

cause display of a field selection interface comprising a first area displaying the first example event with markups indicating (i) user selected tokens from text within the first example event and (ii) correspondence with the fields;

configure the field extraction rule to extract the user selected tokens from the first example event as the values for the fields; and

cause display, on a second area of the field selection interface, of a set of the events from the data source and preview extracted values for the fields, corresponding to the user selected tokens, resulting from applying the field extraction rule to the set of the events.

22. The one or more computer-storage media of claim 21, wherein the wizard is configured to provide a progress line indicating progress through a structured sequence from selecting the user selection of the source type, to selecting the user selection of the first example event, to selecting the user selected tokens, and concluding with saving the configured extraction rule.

23. The one or more computer-storage media of claim 21, wherein the wizard is configured to reduce a number of steps of the wizard by borrowing context from a state existing when the wizard is invoked.

24. The one or more computer-storage media of claim 21, wherein the wizard is configured to identify the user selection of the source type and the user selection of the first example event by recognizing that the source type and the first example event have been selected, while browsing with some other tool than the wizard, when the wizard is invoked.

25. The one or more computer-storage media of claim 21, wherein the wizard is configured to automatically configure the field extraction rule and provide a manual editor configured to accept and implement direct edits to the automatically configured field extraction rule.

26. The one or more computer-storage media of claim 21, wherein the preview extracted values for the fields comprise markups of text on the display of the set of the events from the data source.

27. The one or more computer-storage media of claim 21, wherein the preview extracted values for the fields are distinct from the display of the set of the events from the data source, and wherein the preview extracted values are arranged in columns.

28. The one or more computer-storage media of claim 21, wherein the wizard is configured to present the preview extracted values with an appearance based on a user selection between at least one of markups of text on the display of the set of the events or arranged in columns distinct from the display of the set of the events.

29. The one or more computer-storage media of claim 21, wherein the wizard is configured to receive a selection of a portion of text within a second example event, from the set of the events, and, based thereon, configure the field extraction rule as a more inclusive field extraction rule configured to extract both the portion of text from the second example event and the selected tokens from the first example event.

30. The one or more computer-storage media of claim 21, wherein the wizard is configured to accept a selection naming and saving the field extraction rule.