Application of search policies to searches on event data stored in persistent data structures
First Claim
Patent Images
1. A method, comprising:
- receiving raw data from one or more sources in an information technology environment;
creating a plurality of searchable events based on the raw data by segmenting the raw data into searchable events, each searchable event includes at least a portion of the segmented raw data;
associating a time stamp with each event in the plurality of searchable events;
indexing each time stamped event in the plurality of searchable events;
creating two or more time-based persistent data structures for storing the plurality of searchable events that save the segmented raw data of each time stamped event in the plurality of searchable events and allow application of time-based search phrases across the segmented raw data in the plurality of searchable events, wherein each persistent data structure corresponds to a specific time interval, wherein events stored in a particular persistent data structure have associated time stamps that fall within a particular time interval corresponding to the particular persistent data structure;
searching events in the two or more persistent data structures according to a time-based search phrase;
parsing the time based search phrase into multiple sub-searches, wherein sub-searches of the time-based search phrase are applied sequentially to two or more particular persistent data structures of the two or more persistent data structures, wherein the two or more particular persistent data structures store events having time stamps that fall within a time interval specified by the time-based search phrase, wherein sub-searches are not applied to any subsequent particular persistent data structures upon obtaining a sufficient amount of search results from sub-searches applied to one or more previous particular persistent data structures of the two or more particular persistent data structures.
1 Assignment
0 Petitions
Accused Products
Abstract
Methods and apparatus consistent with the invention provide the ability to organize, index, search, and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is organized into discrete events with normalized time stamps and the events are indexed by time and keyword. A search is received and relevant event information is retrieved based in whole or in part on the time indexing mechanism, keyword indexing mechanism, or statistical indices calculated at the time of the search.
115 Citations
18 Claims
-
1. A method, comprising:
-
receiving raw data from one or more sources in an information technology environment; creating a plurality of searchable events based on the raw data by segmenting the raw data into searchable events, each searchable event includes at least a portion of the segmented raw data; associating a time stamp with each event in the plurality of searchable events; indexing each time stamped event in the plurality of searchable events; creating two or more time-based persistent data structures for storing the plurality of searchable events that save the segmented raw data of each time stamped event in the plurality of searchable events and allow application of time-based search phrases across the segmented raw data in the plurality of searchable events, wherein each persistent data structure corresponds to a specific time interval, wherein events stored in a particular persistent data structure have associated time stamps that fall within a particular time interval corresponding to the particular persistent data structure; searching events in the two or more persistent data structures according to a time-based search phrase; parsing the time based search phrase into multiple sub-searches, wherein sub-searches of the time-based search phrase are applied sequentially to two or more particular persistent data structures of the two or more persistent data structures, wherein the two or more particular persistent data structures store events having time stamps that fall within a time interval specified by the time-based search phrase, wherein sub-searches are not applied to any subsequent particular persistent data structures upon obtaining a sufficient amount of search results from sub-searches applied to one or more previous particular persistent data structures of the two or more particular persistent data structures. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. One or more non-transitory computer-readable storage media, storing one or more sequences of instructions, which when executed by one or more processors cause performance of:
-
receiving raw data from one or more sources in an information technology environment; creating a plurality of searchable events based on the raw data by segmenting the raw data into searchable events, each searchable event includes at least a portion of the segmented raw data; associating a time stamp with each event in the plurality of searchable events; indexing each time stamped event in the plurality of searchable events; creating two or more time-based persistent data structures for storing the plurality of searchable events that save the segmented raw data of each time stamped event in the plurality of searchable events and allow application of time-based search phrases across the segmented raw data in the plurality of searchable events, wherein each persistent data structure corresponds to a specific time interval, wherein events stored in a particular persistent data structure have associated time stamps that fall within a particular time interval corresponding to the particular persistent data structure; searching events in the two or more persistent data structures according to a time-based search phrase; parsing the time based search phrase into multiple sub-searches, wherein sub-searches of the time-based search phrase are applied sequentially to two or more particular persistent data structures of the two or more persistent data structures, wherein the two or more particular persistent data structures store events having time stamps that fall within a time interval specified by the time-based search phrase, wherein sub-searches are not applied to any subsequent particular persistent data structures upon obtaining a sufficient amount of search results from sub-searches applied to one or more previous particular persistent data structures of the two or more particular persistent data structures. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. An apparatus, comprising:
-
a raw data receiver, implemented at least partially in hardware, that receives raw data from one or more sources in an information technology environment; an event creator, implemented at least partially in hardware, that creates a plurality of searchable events based on the raw data by segmenting the raw data into searchable events, each searchable event includes at least a portion of the segmented raw data; a time stamp processor, implemented at least partially in hardware, that associates a time stamp with each event in the plurality of searchable events; an event indexer, implemented at least partially in hardware, that indexes each time stamped event in the plurality of searchable events; a persistent data structure creation device, implemented at least partially in hardware, that creates two or more time-based persistent data structures for storing the plurality of searchable events that save the segmented raw data of each time stamped event in the plurality of searchable events and allow application of time-based search phrases across the segmented raw data in the plurality of searchable events, wherein events stored in a particular persistent data structure have associated time stamps that fall within a particular time interval corresponding to the particular persistent data structure; an event search device, implemented at least partially in hardware, that searches events in the two or more persistent data structures according to a time-based search phrase; wherein the event search device parses the time based search phrase into multiple sub-searches, wherein sub-searches of the time-based search phrase are applied sequentially to two or more particular persistent data structures of the two or more persistent data structures, wherein the two or more particular persistent data structures store events having time stamps that fall within a time interval specified by the time-based search phrase, wherein sub-searches are not applied to any subsequent particular persistent data structures upon obtaining a sufficient amount of search results from sub-searches applied to one or more previous particular persistent data structures of the two or more particular persistent data structures. - View Dependent Claims (16, 17, 18)
-
Specification