Generation of a data model applied to object queries

• US 9,589,012 B2
• Filed: 07/31/2015
• Issued: 03/07/2017
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Neels, Alice Emily, Ganapathi, Archana Sulochana, Robichaud, Marc Vincent, Sorkin, Stephen Phillip, Zhang, Steve Yu
• Primary Examiner(s): NGUYEN, PHONG H
• Application Number:
  US14/815,884
• Publication Number:
  US 20150339344A1
• Time in Patent Office: 585 Days
• Field of Search: 707/765, 707/779
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/186 Templates

Abstract

Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.

Claims

1. A computer-implemented method, comprising:

receiving from a user a selection of an object among one or more objects included in a data model, the selection made through an object-selection interface;

retrieving from computer memory a previously stored object definition that corresponds to the selected object, the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields, each field defined by an extraction rule or regular expression that locates the field in the raw machine data and can be used to extract a field value from the field location from the raw machine data in each event in a subset of the set of time stamped events, each extraction rule or regular expression operating on the raw machine data in an event without modifying the event'"'"'s raw machine data; and

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result based at least in part on the data reflecting the activity of the information technology environment.

2. The method of claim 1, wherein the result is indicative of a performance of an information technology environment.

3. The method of claim 1, wherein the result is indicative of a security of an information technology environment.

4. The method of claim 1, wherein the result is based on an aggregate of values for a field included in the object schema.

5. The method of claim 1, further comprising causing display of the result.

6. The method of claim 1, further comprising causing display of the result in a table.

7. The method of claim 1, further comprising causing display of the result in a graphical visualization.

8. The method of claim 1, further comprising causing an alert or notification based on the result.

9. The method of claim 1, further comprising causing execution of an action based on the result.

10. The method of claim 1, wherein the selected object is a root object.

11. The method of claim 1, wherein the selected object is a child object.

12. The method of claim 1, wherein the selected object is a root object whose object query provides for broader search criteria than a child object query of a child object that is also included in the data model and is selectable through the object-selection interface.

13. The method of claim 1, wherein the selected object is a child object whose object query provides for narrower search criteria than a root object query of a root object that is also included in the data model and is selectable through the object-selection interface.

14. The method of claim 1, wherein the selected object is a root object, and the data model further comprises at least one child object corresponding to a child object definition that is stored in computer memory and that includes:

a child object query that provides for narrower search criteria than a root object query such that, when the child object query is executed against the time stamped events in the data store, the child object query returns a second set of time stamped events that is a subset of the set of time stamped events generated by executing the root object query against the time stamped events in the data store; and

a child object schema identifying a second set of one or more fields, each field defined by an extraction rule or regular expression that can be used to extract a value for the field from each event in a subset of the second set of time stamped events, each extraction operating on the raw machine data in an event without modifying the event'"'"'s raw machine data.

15. The method of claim 1, wherein the object-selection interface is graphical and includes a hierarchical display of references to root and child objects.

16. The method of claim 1, further comprising:

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query.

17. The method of claim 1, wherein the search query includes event-filtering criteria different than filtering criteria of the object query, and wherein the method further comprises:

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query and specify the event-filtering criteria that is different than the filtering criteria of the object query.

18. The method of claim 1, wherein the search query produces the result based at least in part on an aggregate of values for a field included in the object schema, and wherein the method further comprises:

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query and specify the aggregate.

19. The method of claim 1, further comprising:

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object; and

receiving through the pivot graphical user interface a selection of a graphical visualization format for displaying the result.

20. The method of claim 1, wherein the search query is produced manually by a user.

21. The method of claim 1, wherein the search query is produced by a user through a pivot graphical user interface.

22. The method of claim 1, wherein the search query is selected by a user from a displayed list of pre-defined queries.

23. A non-transitory computer readable storage medium impressed with computer program instructions that, when executed on a processor, implement a method comprising:

receiving from a user a selection of an object among one or more objects included in a data model, the selection made through an object-selection interface;

retrieving from computer memory a previously stored object definition that corresponds to the selected object, the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields, each field defined by an extraction rule or regular expression that locates the field in the raw machine data and can be used to extract a field value from the field location from the raw machine data in each event in a subset of the set of time stamped events, each extraction rule or regular expression operating on the raw machine data in an event without modifying the event'"'"'s raw machine data; and

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result based at least in part on the data reflecting the activity of the information technology environment.

24. The computer readable medium of claim 23, wherein the result is indicative of a performance of an information technology environment or a security of an information technology environment.

25. The computer readable medium of claim 23, implementing the method further comprising:

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query.

26. The computer readable medium of claim 23, wherein the search query is produced by a user through a pivot graphical user interface.

27. A system including one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:

receiving from a user a selection of an object among one or more objects included in a data model, the selection made through an object-selection interface;

retrieving from computer memory a previously stored object definition that corresponds to the selected object, the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields, each field defined by an extraction rule or regular expression that locates the field in the raw machine data and can be used to extract a field value from the field location from the raw machine data in each event in a subset of the set of time stamped events, each extraction rule or regular expression operating on the raw machine data in an event without modifying the event'"'"'s raw machine data; and

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result based at least in part on the data reflecting the activity of the information technology environment.

28. The system of claim 27, wherein the result is indicative of a performance of an information technology environment or a security of an information technology environment.

29. The system of claim 27, further configured to:

populate a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query.

30. The computer device of claim 27, wherein the search query is produced by a user through a pivot graphical user interface.