Data Model Selection and Application Based on Data Sources

• US 20170139983A1
• Filed: 01/31/2017
• Published: 05/18/2017
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: NEELS, ALICE EMILY, GANAPATHI, ARCHANA SULOCHANA, ROBICHAUD, MARC VINCENT, SORKIN, STEPHEN PHILLIP, ZHANG, STEVE YU
• Granted Patent: US 10,169,405 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/186 Templates

Abstract

Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.

Claims

1. A method, comprising:

selecting one or more data models among a plurality of data models based on data being analyzed from a specific data source among a plurality of data sources, the one or more data models represent a view and/or perspective of the data associated with the specific data source, the data comprised of a plurality of time-stamped, searchable events, each event in the a plurality of time-stamped, searchable events including a portion of raw machine data reflecting activity in an information technology environment;

causing display, in a graphical user interface, of a representation of one or more objects that are included in the one or more data models;

receiving a selection of a representation of an object among the representation of the one or more objects via the graphical user interface;

based on the selected object representation, retrieving a query and a schema associated with the selected object representation;

retrieving a set of time-stamped, searchable events from the data using the query; and

extracting field values from one or more fields, identified by the object schema, in the portions of raw machine data in the set of time-stamped, searchable events;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is indicative of a performance of an information technology environment.

3. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is indicative of a security of an information technology environment.

4. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is based on an aggregate of values for a field included in the object schema.

5. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

causing display of the result.

6. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

causing display of the result in a table.

7. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

causing display of the result in a graphical visualization.

8. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

causing an alert or notification based on the result.

9. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

causing execution of an action based on the result.

10. The method of claim 1, wherein the selected object is a root object.

11. The method of claim 1, wherein the selected object is a child object.

12. The method of claim 1, wherein the selected object representation refers to a root object whose query provides for broader search criteria than a query of a child object that is also included in the data model and is selectable via the graphical user interface.

13. The method of claim 1, wherein the selected object representation refers to a child object whose query provides for narrower search criteria than a query of a root object that is also included in the data model and is selectable through the object-selection interface.

14. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

populating a second graphical user interface with data-manipulation controls that correspond to the set of one or more fields identified in the schema for the selected object, the second graphical user interface enables a user to develop the search query.

15. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the search query includes event-filtering criteria different than filtering criteria of the query;

populating a second graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the second graphical user interface enables a user to develop the search query and specify the event-filtering criteria that is different than the filtering criteria of the query.

16. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the search query produces the result based at least in part on an aggregate of values for a field included in the object schema;

populating a second graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the second graphical user interface enables a user to develop the search query and specify the aggregate.

17. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

populating a second graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the schema for the selected object;

receiving through the second graphical user interface a selection of a graphical visualization format for displaying the result.

18. The method of claim 1, further comprising:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the search query is selected by a user from a displayed list of pre-defined queries.

19. One or more non-transitory computer-readable storage media, storing one or more sequences of instructions, which when executed by one or more processors cause performance of:

selecting one or more data models among a plurality of data models based on data being analyzed from a specific data source among a plurality of data sources, the one or more data models represent a view and/or perspective of the data associated with the specific data source, the data comprised of a plurality of time-stamped, searchable events, each event in the a plurality of time-stamped, searchable events including a portion of raw machine data reflecting activity in an information technology environment;

causing display, in a graphical user interface, of a representation of one or more objects that are included in the one or more data models;

receiving a selection of a representation of an object among the representation of the one or more objects via the graphical user interface;

based on the selected object representation, retrieving a query and a schema associated with the selected object representation;

retrieving a set of time-stamped, searchable events from the data using the query; and

extracting field values from one or more fields, identified by the object schema, in the portions of raw machine data in the set of time-stamped, searchable events.

20. The one or more non-transitory computer-readable storage media as recited in claim 19, wherein the one or more sequences of instructions, when executed by the one or more processors cause further performance of:

executing a search query across the extracted field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is indicative of a performance of an information technology environment.

23. A non-transitory computer readable storage medium impressed with computer program instructions that, when executed on a processor, implement a method comprising:

receiving from a user a selection of an object among one or more objects included in a data model, the selection made through an object-selection interface;

retrieving from computer memory a previously stored object definition that corresponds to the selected object, the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields, each field defined by an extraction rule or regular expression that locates the field in the raw machine data and can be used to extract a field value from the field location from the raw machine data in each event in a subset of the set of time stamped events, each extraction rule or regular expression operating on the raw machine data in an event without modifying the event'"'"'s raw machine data; and

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result based at least in part on the data reflecting the activity of the information technology environment.

24. The computer readable medium of claim 23, wherein the result is indicative of a performance of an information technology environment or a security of an information technology environment.

25. The computer readable medium of claim 23, implementing the method further comprising:

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query.

26. The computer readable medium of claim 23, wherein the search query is produced by a user through a pivot graphical user interface.

27. A system including one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:

receiving from a user a selection of an object among one or more objects included in a data model, the selection made through an object-selection interface;

retrieving from computer memory a previously stored object definition that corresponds to the selected object, the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields, each field defined by an extraction rule or regular expression that locates the field in the raw machine data and can be used to extract a field value from the field location from the raw machine data in each event in a subset of the set of time stamped events, each extraction rule or regular expression operating on the raw machine data in an event without modifying the event'"'"'s raw machine data; and

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result based at least in part on the data reflecting the activity of the information technology environment.

28. The system of claim 27, wherein the result is indicative of a performance of an information technology environment or a security of an information technology environment.

29. The system of claim 27, further configured to:

populate a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the selected object, and wherein the pivot graphical user interface enables a user to develop the search query.

30. The computer device of claim 27, wherein the search query is produced by a user through a pivot graphical user interface.