System for storage, querying, and analysis of time series data

• US 9,152,671 B2
• Filed: 12/17/2012
• Issued: 10/06/2015
• Est. Priority Date: 12/17/2012
• Status: Active Grant

• Current Assignee: GE Digital Holdings LLC (GE Aerospace)
• Original Assignee: General Electric Company
• Inventors: Lin, Jerry, Aggour, Kareem Sherif, Courtney, Brian Scott, Interrante, John Alan, LaComb, Christina Ann, Mathur, Sunil, McConnell, Christopher Thomas, Snell, Quinn
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/716,554
• Publication Number:
  US 20140172866A1
• Time in Patent Office: 1,023 Days
• Field of Search: 707/746, 707/999.107, 707/999.002, 707/741
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A system for storing time series data includes an ingester that prepares metadata indices associated with blocks of incoming time series data and stores the blocks of data in a time series database and the indices in a separate index database. The time series database distributes storage of the data blocks among multiple data nodes. A query layer receives queries and uses the index database to determine which data blocks are needed to process the query, and then requests only those data blocks from the time series database. Processing of the query is performed within the time series database only on those data nodes that contain relevant data, and partial results are passed to an output layer for formation into a final query result.

Claims

1. A system for time series data storage, querying and analysis, comprising at least one computing device to execute the function of:

a data generator that produces time stamped data associated with the behavior of a plurality of assets;

an ingester configured to receive the time stamped data from the data generator, wherein the ingester configured to read the received time stamped data and to create a data block and an index associated with the received time stamped data;

an index database configured to receive and store the index received from the ingester;

a time series database configured to receive the data blocks from the ingester, wherein the time series database storing the data blocks across a plurality of computing devices, and;

a query layer configured to;

receive a query that specifies criteria defining a set of data retrieved from the system and an analysis performed on that data;

request from the index database the indices associated with the data blocks stored in the time series database needed to evaluate the query;

prepare a sub-query that produces appropriate data matching the criteria, the sub-query including the criteria and a logical operation performed on data matching the criteria; and

send the sub-query to each of the computing devices that corresponds to the data blocks identified in the request from the index database;

an evaluator running on each of the plurality of computing devices, the evaluator configured to;

receive the sub-query from the query layer to evaluate the criteria specified in the sub-query with respect to the data blocks stored on the same computing device as the evaluator to select a subset of data; and

perform the logical operation specified in the sub-query on the subset of data to produce a sub-result; and

an output handler configured to receive the sub-results produced in response to each sub-query and combine them into a query result.

2. The system of claim 1, wherein the data generator comprises a plurality of sensors measuring operational parameters of the plurality of assets in real time.

3. The system of claim 1, wherein the data generator comprises a process that converts existing stored time series data from an external database for storage within the system.

4. The system of claim 1, wherein the index describes metadata associated with the time stamped data.

5. The system of claim 4, wherein the metadata includes at least one of the following types of information:

the asset generating the data in the data block;

the sensor type associated with the data in the data block;

an asset profile associated with the data in the data block;

a time range covered by the data in the data block;

a query associated with the data in the data block;

an owner associated with the data in the data block; and

a security setting associated with the data in the data block.

6. The system of claim 1, wherein the time series database comprises:

a plurality of data nodes, wherein each data node consisting of one of the plurality of computing devices, on which specific data blocks are stored;

a control node configured to choose which data node stores in each data block and to record which data blocks are stored on each of the plurality of data nodes; and

wherein the control node and each of the plurality of data nodes run on a separate one of the plurality of computing devices.

7. The system of claim 1, wherein the time series database comprises:

a plurality of data nodes, wherein each data node consisting of one of the plurality of computing devices, on which specific data blocks are stored;

a control node configured to choose which data node stores in each data block and to record which data blocks are stored on each of the plurality of data nodes, and

wherein an evaluator is running on each data node of the time series database.

8. The system of claim 1, wherein the time series database comprises:

a plurality of data nodes, wherein each data node consisting of one of the plurality of computing devices, on which specific data blocks are stored;

a control node configured to choose which data node stores in each data block and to record which data blocks are stored on each of the plurality of data nodes, and

wherein the control node is configured to assign storage of each data block to multiple data nodes.

9. The system of claim 1, wherein the output handler is further configured to forward the query result to the ingester for storage within the system.

10. The system of claim 1, wherein the output handler is further configured to forward the query results to an external system.

11. The system of claim 1, wherein the index database is running on one of the plurality of computing devices.

12. The system of claim 1, wherein the index database is running on the same one of the plurality of computing devices as the control node of the time series database.

13. The system of claim 1, wherein the query layer is running on one of the plurality of computing devices.

14. A system for time series data storage, querying and analysis, comprising at least one computing device to execute the function of:

a data generator that produces time stamped data associated with the behavior of a plurality of assets;

an ingester configured to receive the time stamped data from the data generator, wherein the ingester configured to read the received time stamped data and to create a plurality of data blocks each comprising time stamped data having a set of shared metadata and an index associated with each of the plurality of data blocks;

an index database configured to receive and store the index received from the ingester;

a time series database configured to receive the data blocks from the ingester, wherein the time series database comprising a plurality of data nodes, wherein each data node configured to store data blocks and each data node comprising an evaluator; and

a control node configured to choose which data node stores in each data block and to record which data blocks are stored on each of the plurality of data nodes;

a query layer configured to;

receive a query that specifies criteria defining a set of data retrieved from the system and an analysis performed on that data;

request from the index database the indices associated with the data blocks stored in the time series database needed to evaluate the query;

prepare a sub-query that produces appropriate data matching the criteria, the sub-query including the criteria and a logical operation performed on data matching the criteria; and

send the sub-query to each data node that corresponds to the data blocks identified in the request from the index database;

an evaluator on each data node that corresponds to the data blocks, wherein the evaluator configured to;

receive the sub-query from the query layer;

evaluate the criteria specified in the sub-query with respect to the data blocks stored on the same data node as the evaluator to select a subset of data; and

perform the logical operation specified in the sub-query on the subset of data to produce a sub-result; and

an output handler configured to receive the sub-results produced in response to each sub-query and combine them into a query result.

15. The system of claim 14, wherein the data generator comprises a process that converts existing stored time series data from an external database for storage within the system.

16. The system of claim 14, wherein the index describes the set of shared metadata and wherein the set of shared metadata comprises at least one of the following types of information:

the asset generating the data in the data block;

the sensor type associated with the data in the data block;

an asset profile associated with the data in the data block;

a time range covered by the data in the data block;

a query associated with the data in the data block;

an owner associated with the data in the data block; and

a security setting associated with the data in the data block.

17. The system of claim 14, wherein the control node is configured to assign storage of each data block to multiple data nodes.

18. The system of claim 14, wherein the output handler is further configured to forward the query result to the ingester for storage within the system.

19. The system of claim 14, wherein the logical operation of the sub-query is based upon the analysis requested in the query.