SYSTEM AND METHOD FOR STORAGE, QUERYING, AND ANALYSIS SERVICE FOR TIME SERIES DATA
First Claim
1. A method of offering a service for storage and querying of real-time asset operational data, comprising the steps of:
- providing a data pipe, comprising a connection through which data flows from a data source to an ingester, the ingester configured to perform the steps of;
receiving time stamped data from the data generator;
reading the received time stamped data;
creating a data block and an index associated with the time stamped data;
storing the index in an index database; and
storing the data block in a time series database in which the time series database stores the data blocks across a plurality of computing devices;
providing a query pipe, comprising a connection through which a query flows from a requestor to a query layer, the query layer configured to perform the steps of;
receiving a query that specifies criteria defining a set of data to be retrieved from the service;
requesting from the index database the indices associated with the data blocks stored in the time series database that are needed to evaluate the query;
preparing a sub-query that will produce appropriate data matching the criteria, the sub-query including the criteria and a logical operation to be performed on data matching the criteria; and
sending the sub-query to an evaluator running on each of the computing devices that corresponds to the data blocks identified in the requesting step, above, the evaluator configured to perform the steps of;
receiving the sub-query from the query layer;
evaluating the criteria specified in the sub-query with respect to the data blocks stored on the same computing devices as the evaluator in order to select a subset of data;
performing the logical operation specified in the sub-query on the subset of data to produce a sub-result; and
returning the sub-result to an output handler;
receive the sub-result from the evaluators specified in the sending step, above;
providing a result pipe, comprising a connection through which a result flows from the output handler to the requestor, the output handler configured to perform the steps of;
receiving the sub-results produced in response to each sub-query;
combining the sub-results into a query result; and
returning the result to the requestor.
2 Assignments
0 Petitions
Accused Products
Abstract
A service for storing time series data provides a data pipe for receiving time series data, a query pipe for making requests to the service, and a result pipe for receiving output from the service. Data sent to the query pipe is processed by an ingester that prepares metadata indices associated with blocks of incoming time series data and stores the blocks of data in a time series database and the indices in a separate index database. A query layer receives queries from the query pipe and uses the index database to determine which data blocks are needed to process the query, and then requests only those data blocks from the time series database. Processing of the query is performed within the time series database only on those data nodes that contain relevant data, and partial results are passed to an output layer for formation into a final query result which is sent out by the results pipe.
-
Citations
15 Claims
-
1. A method of offering a service for storage and querying of real-time asset operational data, comprising the steps of:
-
providing a data pipe, comprising a connection through which data flows from a data source to an ingester, the ingester configured to perform the steps of; receiving time stamped data from the data generator; reading the received time stamped data; creating a data block and an index associated with the time stamped data; storing the index in an index database; and storing the data block in a time series database in which the time series database stores the data blocks across a plurality of computing devices; providing a query pipe, comprising a connection through which a query flows from a requestor to a query layer, the query layer configured to perform the steps of; receiving a query that specifies criteria defining a set of data to be retrieved from the service; requesting from the index database the indices associated with the data blocks stored in the time series database that are needed to evaluate the query; preparing a sub-query that will produce appropriate data matching the criteria, the sub-query including the criteria and a logical operation to be performed on data matching the criteria; and sending the sub-query to an evaluator running on each of the computing devices that corresponds to the data blocks identified in the requesting step, above, the evaluator configured to perform the steps of; receiving the sub-query from the query layer; evaluating the criteria specified in the sub-query with respect to the data blocks stored on the same computing devices as the evaluator in order to select a subset of data; performing the logical operation specified in the sub-query on the subset of data to produce a sub-result; and returning the sub-result to an output handler; receive the sub-result from the evaluators specified in the sending step, above; providing a result pipe, comprising a connection through which a result flows from the output handler to the requestor, the output handler configured to perform the steps of; receiving the sub-results produced in response to each sub-query; combining the sub-results into a query result; and returning the result to the requestor. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
Specification