Data model selection and application based on data sources

• US 10,169,405 B2
• Filed: 01/31/2017
• Issued: 01/01/2019
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Splunk Inc.
• Original Assignee: Splunk Inc.
• Inventors: Neels, Alice Emily, Ganapathi, Archana Sulochana, Robichaud, Marc Vincent, Sorkin, Stephen Phillip, Zhang, Steve Yu
• Primary Examiner(s): Nguyen, Phong H
• Application Number:
  US15/421,415
• Publication Number:
  US 20170139983A1
• Time in Patent Office: 700 Days
• Field of Search: 707765, 707779
• CPC Class Codes: G06F 16/2425 Iterative querying; Query formulation based on the results of a preceding query
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 40/186 Templates

Abstract

Embodiments include generating data models that may give semantic meaning for unstructured or structured data that may include data generated and/or received by search engines, including a time series engine. A method includes generating a data model for data stored in a repository. Generating the data model includes generating an initial query string, executing the initial query string on the data, generating an initial result set based on the initial query string being executed on the data, determining one or more candidate fields from one or results of the initial result set, generating a candidate data model based on the one or more candidate fields, iteratively modifying the candidate data model until the candidate data model models the data, and using the candidate data model as the data model.

Claims

1. A method, comprising:

selecting one or more data models among a plurality of data models based on data being analyzed from a specific data source among a plurality of data sources, the one or more data models representing a view of the data associated with the specific data source, the data comprising a plurality of time-stamped, searchable events, each event in the plurality of time-stamped, searchable events including a portion of unstructured raw machine data reflecting activity in an information technology environment;

causing display, in a graphical user interface, of a representation of one or more objects that are included in the one or more data models;

receiving a selection of a first object representation of a first object among the representation of the one or more objects via the graphical user interface;

based on the first object representation, retrieving, from computer memory, a previously stored object query and an object schema associated with the first object representation;

retrieving a set of time-stamped, searchable events from the data using the object query; and

extracting first field values from one or more fields, identified by the object schema, in portions of unstructured raw machine data in the set of time-stamped, searchable events;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is indicative of a performance of the information technology environment.

3. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is indicative of a security of the information technology environment.

4. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is based on an aggregate of values for a field included in the object schema.

5. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment; and

causing display of the result.

6. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment; and

causing display of the result in a table.

7. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment; and

causing display of the result in a graphical visualization.

8. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment; and

causing an alert or notification based on the result.

9. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment; and

causing execution of an action based on the result.

10. The method of claim 1, wherein the first object is a root object.

11. The method of claim 1, wherein the first object is a child object.

12. The method of claim 1, wherein the first object representation refers to a root object whose object query provides for broader search criteria than an object query of a child object that is also included in the data model and is selectable via the graphical user interface.

13. The method of claim 1, wherein the first object representation refers to a child object whose object query provides for narrower search criteria than an object query of a root object that is also included in the data model and is selectable through the graphical user interface.

14. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment; and

populating a second graphical user interface with data-manipulation controls that correspond to the one or more fields identified in the object schema for the first object, wherein the second graphical user interface enables a user to modify the search query via the data-manipulation controls.

15. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the search query includes event-filtering criteria different than filtering criteria of the object query; and

populating a second graphical user interface with data-manipulation controls that correspond to the one or more fields in the object schema for the first object, wherein the second graphical user interface enables a user to modify the event-filtering criteria via the data-manipulation controls.

16. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the search query produces the result based at least in part on an aggregate of values for a field included in the object schema; and

populating a second graphical user interface with data-manipulation controls that correspond to the one or more fields in the object schema for the first object, wherein the second graphical user interface enables a user to modify the search query and specify the aggregate via the data-manipulation controls.

17. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

populating a second graphical user interface with data-manipulation controls that correspond to the one or more fields in the object schema for the first object; and

receiving, through the second graphical user interface, a selection of a graphical visualization format for displaying the result.

18. The method of claim 1, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the search query is selected by a user from a displayed list of pre-defined queries.

19. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to perform the steps of:

selecting one or more data models among a plurality of data models based on data being analyzed from a specific data source among a plurality of data sources, the one or more data models representing a view of the data associated with the specific data source, the data comprising a plurality of time-stamped, searchable events, each event in the plurality of time-stamped, searchable events including a portion of unstructured raw machine data reflecting activity in an information technology environment;

causing display, in a graphical user interface, of a representation of one or more objects that are included in the one or more data models;

receiving a selection of a first object representation of a first object among the representation of the one or more objects via the graphical user interface;

based on the first object representation, retrieving, from computer memory, a previously stored object query and an object schema associated with the first object representation;

retrieving a set of time-stamped, searchable events from the data using the object query; and

extracting first field values from one or more fields, identified by the object schema, in portions of unstructured raw machine data in the set of time-stamped, searchable events.

20. The non-transitory computer-readable storage medium of claim 19, further comprising:

executing a search query across the first field values that produces a result based at least in part on the data reflecting the activity of the information technology environment;

wherein the result is indicative of a performance of the information technology environment.

29. The non-transitory computer-readable storage medium of claim 19, further comprising:

executing a search query across the first field values to generate a result based at least in part on the data reflecting the activity of the information technology environment, wherein the search query includes event-filtering criteria different than filtering criteria of the object query; and

populating a second graphical user interface with data-manipulation controls that correspond to the one or more fields included in the object schema for the first object, wherein the second graphical user interface enables a user to modify the event-filtering criteria via the data-manipulation controls.

30. The non-transitory computer-readable storage medium of claim 19, further comprising:

executing a search query across the first field values to generate a result based at least in part on the data reflecting the activity of the information technology environment, wherein the search query is selected by a user from a displayed list of pre-defined queries.

21. A non-transitory computer readable storage medium including computer program instructions that, when executed on a processor, implement a method comprising:

selecting one or more data models among a plurality of data models based on data being analyzed from a specific data source among a plurality of data sources, the one or more data models representing a perspective of the data associated with the specific data source, the data comprising a plurality of time stamped events, each event in the plurality of time stamped events including a portion of unstructured raw machine data reflecting activity in an information technology environment;

causing display, in an object-selection interface, of one or more objects that are included in the one or more data models;

receiving, from a user via the object-selection interface, a selection of a first object among the one or more objects included in the one or more data models; and

retrieving, from computer memory, a previously stored object definition that corresponds to the first object, wherein the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of unstructured raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields included in the unstructured raw machine data.

22. The computer readable medium of claim 21, further comprising:

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result, wherein the result is indicative of a performance of an information technology environment or a security of an information technology environment.

23. The computer readable medium of claim 21, implementing the method further comprising:

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result; and

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of the one or more fields identified in the object schema for the first object, wherein the pivot graphical user interface enables a user to modify the search query via the data-manipulation controls.

24. The computer readable medium of claim 21, further comprising:

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result, wherein the search query is produced by a user through a pivot graphical user interface.

25. A system including one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement the steps of:

selecting one or more data models among a plurality of data models based on data being analyzed from a specific data source among a plurality of data sources, the one or more data models representing a view of the data associated with the specific data source, the data comprising a plurality of time stamped events, each event in the plurality of time stamped events including a portion of unstructured raw machine data reflecting activity in an information technology environment;

causing display, in an object-selection interface, of one or more objects that are included in the one or more data models;

receiving, from a user via the object-selection interface, a selection of a first object among the one or more objects included in the one or more data models;

retrieving, from computer memory, a previously stored object definition that corresponds to the first object, wherein the previously stored object definition includes;

an object query that, when executed, retrieves a set of time stamped events from a data store on a computing device, each event including a portion of unstructured raw machine data reflecting activity in an information technology environment; and

an object schema identifying a set of one or more fields included in the unstructured raw machine data.

26. The system of claim 25, further comprising:

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result, wherein the result is indicative of a performance of an information technology environment or a security of an information technology environment.

27. The system of claim 25, further comprising:

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result; and

populating a pivot graphical user interface with data-manipulation controls that correspond to the set of one or more fields in the object schema for the first object, wherein the pivot graphical user interface enables a user to modify the search query via the data-manipulation controls.

28. The system of claim 25, further comprising:

executing, against events in the data store that meet filtering criteria of the object query, a search query that references only field values that are extracted using the object schema and that produces a result, wherein the search query is produced by a user through a pivot graphical user interface.