Generating Reports from Unstructured Data

• US 20150019537A1
• Filed: 09/30/2014
• Published: 01/15/2015
• Est. Priority Date: 09/07/2012
• Status: Abandoned Application

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Vasan, Sundar, Robichaud, Marc, Neels, Alice, Fishel, Simon, Lamas, Divanny
• Application Number:
  US14/503,335
• Publication Number:
  US 20150019537A1
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

The disclosure relates to certain system and method embodiments for generating reports from unstructured data. In one embodiment, a method can include identifying events matching criteria of an initial search query (each of the events including a portion of raw machine data that is associated with a time), identifying a set of fields, each field defined for one or more of the identified events, causing display of an interactive graphical user interface (GUI) that includes one or more interactive elements enabling a user to define a report for providing information relating to the matching events (each interactive element enabling processing or presentation of information in the matching events using one or more fields in the identified set of fields), receiving, via the GUI, a report definition indicating how to report information relating to the matching events, and generating, based on the report definition, a report including information relating to the matching events.

Claims

1. A method comprising:

identifying events matching criteria of an initial search query, wherein each of the events comprises a portion of raw machine data that is associated with a time;

identifying a set of fields, each field defined for one or more of the identified events, and each field is defined by an extraction rule for extracting a value for each of the one or more identified events from the portion of raw data in the event;

causing display of an interactive graphical user interface (GUI) that includes one or more interactive elements enabling a user to define a report for providing information relating to the matching events, each interactive element enabling processing or presentation of information in the matching events using one or more fields in the identified set of fields;

receiving, via the GUI, a report definition indicating how to report information relating to the matching events; and

generating, based on the report definition, a report comprising information relating to the matching events.

2. The method of claim 1, wherein at least one of the one or more interactive elements of the GUI is configured to enable a user to select a field of the set of fields identified and define one or more reporting criteria based on the selected field.

3. The method of claim 1, wherein at least one of the one or more interactive elements of the GUI comprises a filter element configured to enable a user to select a field of the set of fields identified for use in further filtering the matching events based on values for the field.

4. The method of claim 1, wherein at least one of the one or more interactive elements of the GUI comprises a splitting element configured to enable a user to select a field of the set of fields identified for use in grouping events by values for the field.

5. The method of claim 1, wherein at least one of the one or more interactive elements of the GUI is configured to enable a user to select an aggregate to be determined for a field of the set of fields identified.

6. The method of claim 1, further comprising causing display of the report.

7. The method of claim 1, further comprising causing display of the report in an interactive GUI comprising interactive elements that enable a user to modify the report definition.

8. The method of claim 1, wherein the report comprises some or all of the values for the fields.

9. The method of claim 1, wherein the report comprises a table comprising some or all of the values for the fields.

10. The method of claim 1, wherein the report comprises a table comprising one or more rows and one or more columns, wherein each of the one or more rows corresponds to a unique value for a particular field, and wherein each of the one or more columns corresponds to an aggregate and each row of the column corresponds to a value for the aggregate for the unique value corresponding to the respective row.

11. The method of claim 1, wherein the report comprises an aggregate calculated using data from the matching events.

12. The method of claim 1, wherein the report comprises an aggregate calculated using data from the matching events, and wherein the aggregate comprises at least one of the following:

a count, a sum, an average, a maximum, a minimum, a standard deviation, a list of distinct values, a count of distinct values, a first value, a last value, a duration, or an earliest value, or a latest value.

13. The method of claim 1, wherein the report comprises a visualization of data from the matching events.

14. The method of claim 1, wherein the report comprises a visualization of data from the matching events, wherein the visualization comprises a graphical depiction of at least one of the following:

a table, a chart, a graph, or a gauge.

15. The method of claim 1, wherein the report definition specifies additional filtering criteria for one or more of the set of fields that exists in the matching events, and wherein the report comprises a subset of the matching events identified by filtering the matching events based on the additional filtering criteria.

16. The method of claim 1, wherein the report definition specifies additional filtering criteria for one or more of the set of fields that exists in the matching events, and wherein the report comprises an aggregate calculated using data from a subset of the matching events identified by filtering the matching events based on the additional filtering criteria.

17. The method of claim 1, wherein the report definition specifies additional filtering criteria for one or more of the set of fields that exists in the matching events, and wherein the report comprises a visualization generated using data from a subset of the matching events identified by filtering the matching events based on the additional filtering criteria.

18. The method of claim 1, wherein identifying a set of fields comprises:

using a configuration file to identify fields for one or more of the matching events, wherein the set of fields comprises the fields identified using the configuration file.

19. The method of claim 1, wherein identifying a set of fields comprises:

for each of the matching events, using a corresponding extraction rule to identify values for fields of the event.

20. The method of claim 1, wherein identifying the set of fields comprises automatically selecting fields from fields of the matching events.

21. The method of claim 1, wherein identifying the set of fields comprises identifying one or more fields that exists in at least a threshold percentage of the matching events, wherein the set of fields comprises the one or more fields that exists in at least a threshold percentage of the matching events.

22. The method of claim 1, wherein identifying the set of fields comprises receiving manual selection of the set of fields from fields of the matching events, wherein the identified set of fields comprises the manually selected fields.

23. The method of claim 1, wherein identifying the set of fields comprises:

causing display of a second interactive GUI comprising a listing of fields of the matching events; and

receiving selection of one or more of the fields of the listing of fields, wherein the identified set of fields comprises the one or more fields selected from the listing of fields.

24. The method of claim 1, wherein the initial search query is submitted by a user via a user interface.

25. The method of claim 1, wherein generating a report comprises manipulating raw machine data associated with the matching events.

26. The method of claim 1, further comprising:

storing the report definition in memory, wherein the report definition comprises the initial search query and the set of fields, and information indicating how to report information relating to the matching events.

27. The method of claim 1, further comprising:

storing, in memory, a data model object comprising the initial search query and the set of fields.

28. The method of claim 1, wherein identifying events matching criteria of an initial search query comprises employing late-binding schema.

29. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by a processing device, cause the processing device to perform operations comprising:

identifying events matching criteria of an initial search query, wherein each of the events comprises a portion of raw machine data that is associated with a time;

identifying a set of fields, each field defined for one or more of the identified events, and each field is defined by an extraction rule for extracting a value for each of the one or more identified events from the portion of raw data in the event;

causing display of an interactive graphical user interface (GUI) that includes one or more interactive elements enabling a user to define a report for providing information relating to the matching events, each interactive element enabling processing or presentation of information in the matching events using one or more fields in the identified set of fields;

receiving, via the GUI, a report definition indicating how to report information relating to the matching events; and

generating, based on the report definition, a report comprising information relating to the matching events.

30. A system comprising:

a non-transitory computer readable storage medium comprising program instructions; and

a processing device configured to execute the program instructions to cause;

identifying events matching criteria of an initial search query, wherein each of the events comprises a portion of raw machine data that is associated with a time;

identifying a set of fields, each field defined for one or more of the identified events, and each field is defined by an extraction rule for extracting a value for each of the one or more identified events from the portion of raw data in the event;

causing display of an interactive graphical user interface (GUI) that includes one or more interactive elements enabling a user to define a report for providing information relating to the matching events, each interactive element enabling processing or presentation of information in the matching events using one or more fields in the identified set of fields;

receiving, via the GUI, a report definition indicating how to report information relating to the matching events; and

generating, based on the report definition, a report comprising information relating to the matching events.