DATA MODEL FOR MACHINE DATA FOR SEMANTIC SEARCH

• US 20140074887A1
• Filed: 09/07/2012
• Published: 03/13/2014
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Neels, Alice Emily, Ganapathi, Archana Sulochana, Robichaud, Marc Vincent, Sorkin, Stephen Phillip, Zhang, Steve Yu
• Granted Patent: US 8,788,525 B2
• Time in Patent Office: Days
• US Class Current: 707/779
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/186 Templates

Abstract

Embodiments are directed towards generating data models that may give semantic meaning for unstructured data or structured data that may include data generated and/or received by search engines, including a time series engine. Data models also may be generated to provide semantic meaning to structured data. A data model may be composed of a hierarchical data model objects analogous to an object-oriented programming class hierarchy. Users may employ a data modeling application to produce reports using search objects that may be part of, or associated with the data model. The data modeling application may employ the search object and the data model to generate a query string for searching a data repository to produce a result set. A data modeling application may map the result set data to data model objects that may be used to generate reports.

Claims

1-24. -24. (canceled)

25. A computer implemented method, comprising:

storing a set of time stamped events in a data store on a computing device including one or more processors, wherein the set of events are searchable, and wherein the set of events include unstructured machine data;

generating, by the computing device, a data model that models the set of events, wherein the data model defines a schema for the unstructured data in the set of events without modifying an organization of the unstructured data, wherein the data model includes one or more sub-models, and wherein each sub-model includes criteria for identifying a subset of the set of events corresponding to the sub-model;

displaying a graphical interface that enables selection of one or more sub-models of the data model;

receiving input corresponding to a selection of a sub-model through the graphical interface;

narrowing the set of events that are searchable to a subset of events that correspond to criteria for the selected sub-model; and

generating a search query based on the generated data model.

26. The method of claim 25, wherein a sub-model further includes one or more field definitions, wherein each field definition defines a field that includes at most one value for each event in the subset of events, and wherein the at most one value for the field for each event in the subset is derived from data in each event, and further comprising:

in response to receiving the input corresponding to the selection of the sub-model, enabling search queries on the subset of events that correspond to the selected sub-model using fields defined by the field definitions associated with the selected sub-model.

27. The method of claim 25, wherein a sub-model further includes one or more field definitions, wherein each field definition defines a field that includes at most one value for each event in the subset of events, and wherein the at most one value for the field for each event in the subset is derived from data in each event, and further comprising:

in response to receiving the input corresponding to the selection of the sub-model, enabling search queries on the subset of events that correspond to the selected sub-model using fields defined by the field definitions associated with the selected sub-model;

receiving input corresponding to criteria for one or more fields defined by the field definitions associated with the selected sub-model, wherein the criteria is received through a graphical interface;

generating the search query in a search language designed for accessing the events in the data store, wherein the search query includes the criteria for the one or more fields, and wherein the search query executes only over the subset of events that correspond to the selected sub-model; and

executing the search query.

28. The method of claim 25, wherein the set of events are derived by segmenting the unstructured data and assigning a timestamp to each segment.

29. The method of claim 25, wherein the one or more sub-models are defined in an object-oriented programming language.

30. The method of claim 25, wherein the one or more sub-models are arranged in a hierarchy, wherein sub-models inherit a subset of events from another sub-model in the hierarchy, and wherein each sub-model in a lower level of the hierarchy includes further criteria for identifying a further subset of the subset of the events.

31. The method of claim 25, further comprising:

receiving input corresponding to a selection of a visualization for displaying results identified by the search query, wherein the input is received through a graphical interface; and

displaying the visualization of the results.

32. The method of claim 25, further comprising:

receiving input corresponding to a selection of a visualization for displaying results identified by the search query, wherein the input is received through a graphical interface, and wherein the visualization includes a graph or a chart; and

displaying the visualization of the results.

33. The method of claim 25, further comprising:

receiving input corresponding to a selection of a reporting mode for displaying results identified by the search query, wherein the input is received through a graphical interface; and

displaying the results in the reporting mode.

34. The method of claim 25, further comprising:

receiving input corresponding to a selection of a table format for displaying results identified by the search query, wherein the input is received through a graphical interface; and

displaying the results in the table format.

35. A system, comprising:

a processor, and

a non-transitory computer-readable storage medium containing instructions configured to cause the processor to perform operations including;

storing a set of time stamped events in a data store on a computing device including one or more processors, wherein the set of events are searchable, and wherein the set of events include unstructured machine data;

generating a data model that models the set of events, wherein the data model defines a schema for the unstructured data in the set of events without modifying an organization of the unstructured data, wherein the data model includes one or more sub-models, and wherein each sub-model includes criteria for identifying a subset of the set of events corresponding to the sub-model;

displaying a graphical interface that enables selection of one or more sub-models of the data model;

receiving input corresponding to a selection of a sub-model through the graphical interface;

narrowing the set of events that are searchable to a subset of events that correspond to criteria for the selected sub-model; and

generating a search query based on the generated data model.

36. The system of claim 35, wherein a sub-model further includes one or more field definitions, wherein each field definition defines a field that includes at most one value for each event in the subset of events, and wherein the at most one value for the field for each event in the subset is derived from data in each event, and wherein the non-transitory computer-readable storage medium contains further instructions configured to cause the processor to perform operations including:

in response to receiving the input corresponding to the selection of the sub-model, enabling search queries on the subset of events that correspond to the selected sub-model using fields defined by the field definitions associated with the selected sub-model.

37. The system of claim 35, wherein a sub-model further includes one or more field definitions, wherein each field definition defines a field that includes at most one value for each event in the subset of events, and wherein the at most one value for the field for each event in the subset is derived from data in each event, and wherein the non-transitory computer-readable storage medium contains further instructions configured to cause the processor to perform operations including:

in response to receiving the input corresponding to the selection of the sub-model, enabling search queries on the subset of events that correspond to the selected sub-model using fields defined by the field definitions associated with the selected sub-model;

receiving input corresponding to criteria for one or more fields defined by the field definitions associated with the selected sub-model, wherein the criteria is received through a graphical interface;

generating the search query in a search language designed for accessing the events in the data store, wherein the search query includes the criteria for the one or more fields, and wherein the search query executes only over the subset of events that correspond to the selected sub-model; and

executing the search query.

38. The system of claim 35, wherein the set of events are derived by segmenting the unstructured data and assigning a timestamp to each segment.

39. The system of claim 35, wherein the one or more sub-models are defined in an object-oriented programming language.

40. The system of claim 35, wherein the one or more sub-models are arranged in a hierarchy, wherein sub-models inherit a subset of events from another sub-model in the hierarchy, and wherein each sub-model in a lower level of the hierarchy includes further criteria for identifying a further subset of the subset of the events.

41. The system of claim 35, wherein the non-transitory computer-readable storage medium contains further instructions configured to cause the processor to perform operations including:

receiving input corresponding to a selection of a visualization for displaying results identified by the search query, wherein the input is received through a graphical interface; and

displaying the visualization of the results.

42. The system of claim 35, wherein the non-transitory computer-readable storage medium contains further instructions configured to cause the processor to perform operations including:

receiving input corresponding to a selection of a visualization for displaying results identified by the search query, wherein the input is received through a graphical interface, and wherein the visualization includes a graph or a chart; and

displaying the visualization of the results.

43. The system of claim 35, wherein the non-transitory computer-readable storage medium contains further instructions configured to cause the processor to perform operations including:

receiving input corresponding to a selection of a reporting mode for displaying results identified by the search query, wherein the input is received through a graphical interface; and

displaying the results in the reporting mode.

44. The system of claim 35, wherein the non-transitory computer-readable storage medium contains further instructions configured to cause the processor to perform operations including:

receiving input corresponding to a selection of a table format for displaying results identified by the search query, wherein the input is received through a graphical interface; and

displaying the results in the table format.

45. A computer-program product, tangibly embodied in a non-transitory machine-readable medium, including instructions configured to cause a data processing apparatus to:

store a set of time stamped events in a data store on a computing device including one or more processors, wherein the set of events are searchable, and wherein the set of events include unstructured machine data;

generate a data model that models the set of events, wherein the data model defines a schema for the unstructured data in the set of events without modifying an organization of the unstructured data, wherein the data model includes one or more sub-models, and wherein each sub-model includes criteria for identifying a subset of the set of events corresponding to the sub-model;

display a graphical interface that enables selection of one or more sub-models of the data model;

receive input corresponding to a selection of a sub-model through the graphical interface;

narrow the set of events that are searchable to a subset of events that correspond to criteria for the selected sub-model; and

generate a search query based on the generated data model.

46. The computer-program product of claim 45, wherein a sub-model further includes one or more field definitions, wherein each field definition defines a field that includes at most one value for each event in the subset of events, and wherein the at most one value for the field for each event in the subset is derived from data in each event, and further including instructions configured to cause the data processing apparatus to:

in response to receiving the input corresponding to the selection of the sub-model, enable search queries on the subset of events that correspond to the selected sub-model using fields defined by the field definitions associated with the selected sub-model.

47. The computer-program product of claim 45, wherein a sub-model further includes one or more field definitions, wherein each field definition defines a field that includes at most one value for each event in the subset of events, and wherein the at most one value for the field for each event in the subset is derived from data in each event, and further including instructions configured to cause the data processing apparatus to:

in response to receiving the input corresponding to the selection of the sub-model, enable search queries on the subset of events that correspond to the selected sub-model using fields defined by the field definitions associated with the selected sub-model;

receive input corresponding to criteria for one or more fields defined by the field definitions associated with the selected sub-model, wherein the criteria is received through a graphical interface;

generate the search query in a search language designed for accessing the events in the data store, wherein the search query includes the criteria for the one or more fields, and wherein the search query executes only over the subset of events that correspond to the selected sub-model; and

execute the search query.

48. The computer-program product of claim 45, wherein the set of events are derived by segmenting the unstructured data and assigning a timestamp to each segment.

49. The computer-program product of claim 45, wherein the one or more sub-models are defined in an object-oriented programming language.

50. The computer-program product of claim 45, wherein the one or more sub-models are arranged in a hierarchy, wherein sub-models inherit a subset of events from another sub-model in the hierarchy, and wherein each sub-model in a lower level of the hierarchy includes further criteria for identifying a further subset of the subset of the events.

51. The computer-program product of claim 45, further including instructions configured to cause the data processing apparatus to:

receive input corresponding to a selection of a visualization for displaying results identified by the search query, wherein the input is received through a graphical interface; and

display the visualization of the results.

52. The computer-program product of claim 45, further including instructions configured to cause the data processing apparatus to:

receive input corresponding to a selection of a visualization for displaying results identified by the search query, wherein the input is received through a graphical interface, and wherein the visualization includes a graph or a chart; and

display the visualization of the results.

53. The computer-program product of claim 45, further including instructions configured to cause the data processing apparatus to:

receive input corresponding to a selection of a reporting mode for displaying results identified by the search query, wherein the input is received through a graphical interface; and

display the results in the reporting mode.

54. The computer-program product of claim 45, further including instructions configured to cause the data processing apparatus to:

receive input corresponding to a selection of a table format for displaying results identified by the search query, wherein the input is received through a graphical interface; and

display the results in the table format.