Search query processing using operational parameters

• US 10,678,767 B2
• Filed: 08/01/2015
• Issued: 06/09/2020
• Est. Priority Date: 10/05/2006
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Baum, Michael Joseph, Swan, Erik M., Carasso, R. David, Das, Robin Kumar, Greene, Rory, Hall, Bradley, Mealy, Nicholas Christian, Murphy, Brian Philip, Sorkin, Stephen Phillip, Stechert, Andre David
• Primary Examiner(s): Uddin, Mohammed R
• Application Number:
  US14/815,980
• Publication Number:
  US 20150339351A1
• Time in Patent Office: 1,774 Days
• Field of Search: 707746, 707750, 707754
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Methods and apparatus consistent with the invention provide the ability to search and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is organized into discrete events with normalized time stamps and the events are indexed by time and keyword. A search is received and relevant event information is retrieved based in whole or in part on the time indexing mechanism, keyword indexing mechanism, or statistical indices calculated at the time of the search.

Claims

1. A method, comprising:

creating a set of searchable, time stamped events by segmenting raw time series machine data received from at least one data source in an information technology environment into searchable, time stamped events;

processing, by a device, a time-based search phrase by parsing the time-based search phrase having at least two command line portions, a first portion of the at least two command line portions having at least one search phrase to be executed across the set of searchable, time stamped events, and a second portion of the at least two command line portions having at least one data modification operation that specifies processing to be performed by the device on a result data set resulting from executing the search phrase across the set of searchable, time stamped events;

wherein processing the time-based search phrase further comprises;

creating, by the device, the result data set by searching at least a portion of the set of searchable, time stamped events using the at least one search phrase;

creating, by the device, a data modification result by applying the at least one data modification operation to data in the result data set; and

causing display of information relating to the data modification result.

2. The method of claim 1, wherein creating the set of events further comprises filtering the set of searchable, time stamped events to those having time stamps that fall within a user-specified time frame.

3. The method of claim 1, wherein creating the set of events further comprises:

aggregating a plurality of subsets of events that have been created by applying the at least one search parameter to a plurality of different sets of searchable, time stamped events, wherein the different sets of searchable, time stamped events correspond to different time frames.

4. The method of claim 1, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data from the at least one data source, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

5. The method of claim 1, wherein the set of searchable, time stamped events from the at least one data source are parsed from a server log, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed server log.

6. The method of claim 1, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data reflecting transaction activity in an information technology environment, and wherein each event in the set of searchable, time stamped events includes a portion of the machine data.

7. The method of claim 1, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data reflecting recorded measurements, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

8. The method of claim 1, wherein each event in the set of searchable, time stamped events includes a portion of parsed machine data reflecting activity in an information technology environment.

9. The method of claim 1, wherein the search phrase result reflects a performance aspect of the information technology environment.

10. The method of claim 1, wherein the search phrase result reflects a security aspect of the information technology environment.

11. The method of claim 1, wherein each event in the set of searchable, time stamped events includes a portion of parsed machine data reflecting activity in an information technology environment, and wherein the search phrase result reflects a performance aspect of the information technology environment.

12. The method of claim 1, wherein each event in the set of searchable, time stamped events includes a portion of parsed machine data reflecting activity in an information technology environment, and wherein the search phrase result reflects a security aspect of the information technology environment.

13. An apparatus, comprising:

one or more processors; and

a memory storing instructions, which when executed by the one or more processors, causes the one or more processors to;

create a set of searchable, time stamped events by segmenting raw time series machine data received from at least one data source in an information technology environment into searchable, time stamped events;

process a time-based search phrase by parsing the time-based search phrase having at least two command line portions, a first portion of the at least two command line portions having at least one search phrase to be executed across the set of searchable, time stamped events, and a second portion of the at least two command line portions having at least one data modification operation that specifies processing to be performed on a result data set resulting from executing the search phrase across the set of searchable, time stamped events;

wherein process the time-based search phrase further comprises;

create the result data set by searching at least a portion of the set of searchable, time stamped events using the at least one search phrase;

create a data modification result by applying the at least one data modification operation to data in the result data set; and

cause display of information relating to the data modification result.

14. The apparatus of claim 13, wherein the create the set of events further comprises filtering the set of searchable, time stamped events to those having time stamps that fall within a user-specified time frame.

15. The apparatus of claim 13, wherein the create the set of events further comprises also includes aggregating a plurality of subsets of events that have been created by applying the at least one search parameter to a plurality of different sets of searchable, time stamped events, wherein the different sets of searchable, time stamped events correspond to different time frames.

16. The apparatus of claim 13, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data from the at least one data source, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

17. The apparatus of claim 13, wherein the set of searchable, time stamped events from the at least one data source are parsed from a server log, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed server log.

18. The apparatus of claim 13, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data reflecting transaction activity in an information technology environment, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

19. The apparatus of claim 13, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data reflecting recorded measurements, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

20. The apparatus of claim 13, wherein each event in the set of searchable, time stamped events includes a portion of parsed machine data reflecting activity in the information technology environment.

21. The apparatus of claim 13, wherein the search phrase result reflects a performance aspect of the information technology environment.

22. One or more non-transitory computer-readable storage media, storing software instructions, which when executed by one or more processors cause performance of:

creating a set of searchable, time stamped events by segmenting raw time series machine data received from at least one data source in an information technology environment into searchable, time stamped events;

processing, by a device, a time-based search phrase by parsing the time-based search phrase having at least two command line portions, a first portion of the at least two command line portions having at least one search phrase to be executed across the set of searchable, time stamped events, and a second portion of the at least two command line portions having at least one data modification operation that specifies processing to be performed by the device on a result data set resulting from executing the search phrase across the set of searchable, time stamped events;

wherein processing the time-based search phrase further comprises;

creating, by the device, the result data set by searching at least a portion of the set of searchable, time stamped events using the at least one search phrase;

creating, by the device, a data modification result by applying the at least one data modification operation to data in the result data set; and

causing display of information relating to the data modification result.

23. The one or more non-transitory computer-readable storage media of claim 22, wherein creating the set of events further comprises filtering the set of searchable, time stamped events to those having time stamps that fall within a user-specified time frame.

24. The one or more non-transitory computer-readable storage media of claim 22, wherein creating the set of events further comprises:

aggregating a plurality of subsets of events that have been created by applying the at least one search parameter to a plurality of different sets of searchable, time stamped events, wherein the different sets of searchable, time stamped events correspond to different time frames.

25. The one or more non-transitory computer-readable storage media of claim 22, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data from the at least one data source, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

26. The one or more non-transitory computer-readable storage media of claim 22, wherein the set of searchable, time stamped events from the at least one data source are parsed from a server log, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed server log.

27. The one or more non-transitory computer-readable storage media of claim 22, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data reflecting transaction activity in an information technology environment, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

28. The one or more non-transitory computer-readable storage media of claim 22, wherein the set of searchable, time stamped events from the at least one data source are parsed from machine data reflecting recorded measurements, and wherein each event in the set of searchable, time stamped events includes a portion of the parsed machine data.

29. The one or more non-transitory computer-readable storage media of claim 22, wherein each event in the set of searchable, time stamped events includes a portion of parsed machine data reflecting activity in the information technology environment.

30. The one or more non-transitory computer-readable storage media of claim 22, wherein the search phrase result reflects a performance aspect of the information technology environment.