Method for storage, querying, and analysis of time series data

• US 9,152,672 B2
• Filed: 12/17/2012
• Issued: 10/06/2015
• Est. Priority Date: 12/17/2012
• Status: Active Grant

• Current Assignee: GE Digital Holdings LLC (GE Aerospace)
• Original Assignee: General Electric Company
• Inventors: Lin, Jerry, Aggour, Kareem Sherif, Courtney, Brian Scott, Interrante, John Alan, LaComb, Christina Ann, Mathur, Sunil, McConnell, Christopher Thomas, Snell, Quinn
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/716,560
• Publication Number:
  US 20140172867A1
• Time in Patent Office: 1,023 Days
• Field of Search: 707/746, 707/999.107, 707/999.002, 707/741
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A method for performing queries on a distributed time series data storage system is presented. The time series data storage system has a time series database that stores data blocks containing time stamped data across a plurality of computing devices. The system also includes an index database that stores an index associated with the time stamped data in each data block. The method includes the steps of sending a query, requesting indices, returning the indices, preparing a sub-query, forwarding the sub-query to an evaluator, evaluating the sub-query, performing a logical operation on each sub-query'"'"'s result, receiving the sub-results at an output handler, and combining the sub-results.

Claims

1. A method for performing queries on a distributed time series data storage system having a time series database that stores data blocks containing time stamped data across a plurality of computing devices, and an index database that stores an index associated with the time stamped data in each data block, the method comprising:

sending a query to a query layer running on a first computing device, the query specifying criteria that define a set of data retrieved from the time series data storage system and an analysis performed on the set of data;

requesting from the index database the indices associated with the data blocks stored in the time series database needed to evaluate the query;

returning the indices back to the query layer;

preparing a sub-query that produces appropriate data matching the criteria, the sub-query including the criteria and a logical operation performed on the data matching the criteria;

forwarding the sub-query to an evaluator running on each of the plurality of computing devices that are identified in the returned indices as holding data corresponding to the data blocks needed to evaluate the query;

evaluating the criteria specified in the sub-query in each evaluator with respect to the data blocks stored on the same computing device on which the evaluator is running in order to select a subset of data;

performing the logical operation specified in the sub-query in each evaluator on the subset of data generated in that evaluator in the evaluating step above to generate a sub-result;

receiving each sub-result from each evaluator at an output handler; and

combining the sub-result from each evaluator into a query result.

2. The method of claim 1 wherein the time series data storage system further comprises a data generator that produces the time stamped data and an ingester, and wherein the method further comprises:

receiving the time-stamped data from the data generator by the ingester;

creating the data block and the index from the time-stamped data;

sending the data block to the time series database for storage; and

sending the index to the index database for storage.

3. The method of claim 2 wherein the data generator comprises a plurality of sensors measuring operational parameters of a plurality of assets in real time.

4. The method of claim 2 wherein the data generator receives existing time series data from an external database.

5. The method of claim 2 further comprising:

sending the query result to the ingester for storage within the time series database.

6. The method of claim 1 wherein the index describes metadata associated with the time stamped data.

7. The method of claim 6 wherein the metadata includes at least one of the following types of information:

the asset generating the data in the data block;

the sensor type associated with the data in the data block;

an asset profile associated with the data in the data block;

a time range covered by the data in the data block;

a query associated with the data in the data block;

an owner associated with the data in the data block; and

a security setting associated with the data in the data block.

8. The method of claim 1 wherein the time series database comprises:

a plurality of data nodes, each consisting of one of the plurality of computing devices, on which specific data blocks are stored; and

a control node configured to choose which data node will store each data block and to record which data blocks are stored on each of the plurality of data nodes.

9. The method of claim 8, wherein the control node and each of the plurality of data nodes run on a separate one of the plurality of computing devices.

10. The method of claim 8, wherein an evaluator is running on each data node of the time series database.

11. The method of claim 8, wherein control node is configured to assign storage of each data block to multiple data nodes.

12. The method of claim 1, wherein the index database is running on one of the plurality of computing devices.

13. The method of claim 1, wherein the index database is running on the same one of the plurality of computing devices as the control node of the time series database.

14. The method of claim 1, wherein the query layer is running on one of the plurality of computing devices.

15. The method of claim 1, wherein the time stamped data is associated with the behavior of a plurality of assets.

16. A non-transitory computer readable medium containing program instruction for execution on a general purpose computing device, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform queries on a distributed time series data storage system having a time series database that stores data blocks containing time stamped data across a plurality of computing devices, and an index database that stores an index associated with the time stamped data in each data block, the one or more processors performing said queries by carrying out operations comprising:

sending a query to a query layer running on a first computing device, the query specifying criteria that define a set of data retrieved from the time series data storage system and an analysis performed on the set of data;

requesting from the index database the indices associated with the data blocks stored in the time series database needed to evaluate the query;

returning the indices back to the query layer;

preparing a sub-query that produces appropriate data matching the criteria, the sub-query including the criteria and a logical operation performed on the data matching the criteria;

forwarding the sub-query to an evaluator running on each of the plurality of computing devices that are identified in the returned indices as holding data corresponding to the data blocks needed to evaluate the query;

evaluating the criteria specified in the sub-query in each evaluator with respect to the data blocks stored on the same computing device on which the evaluator is running in order to select a subset of data;

performing the logical operation specified in the sub-query in each evaluator on the subset of data generated in that evaluator in the evaluating step above to generate a sub-result;

receiving each sub-result from each evaluator at an output handler; and

combining the sub-result from each evaluator into a query result.

17. The non-transitory computer readable medium of claim 16 wherein the time series data storage system further comprises a data generator that produces the time stamped data and an ingester, and wherein the method further comprises:

receiving the time-stamped data from the data generator by the ingester;

creating the data block and the index from the time-stamped data;

sending the data block to the time series database for storage; and

sending the index to the index database for storage.

18. The non-transitory computer readable medium of claim 16 wherein the time series database comprises:

a plurality of data nodes, each consisting of one of the plurality of computing devices, on which specific data blocks are stored; and

a control node configured to choose which data node will store each data block and to record which data blocks are stored on each of the plurality of data nodes.

19. A non-transitory computer readable medium comprising program instruction for execution on a general purpose computing device, wherein execution of the program instructions by one or more processors of a computer system configures the one or more processors to perform operations comprising:

receive a query at a query layer running on a first computing device, the query specifying criteria that define a set of data retrieved from the time series data storage system and an analysis performed on the set of data;

request from an index database indices associated with data blocks stored in a time series database needed to evaluate the query, the time series database comprising a control node and a plurality of data nodes;

receive the indices back to the query layer;

prepare a sub-query that produces appropriate data matching the criteria, the sub-query comprising the criteria and a logical operation performed on the data matching the criteria;

forward the sub-query to an evaluator running on each of the plurality of data nodes that are identified in the returned indices as holding data corresponding to data blocks needed to evaluate the query;

evaluate the criteria specified in the sub-query in each evaluator with respect to data blocks stored on the same computing device on which the evaluator is running in order to select a subset of data;

perform the logical operation specified in the sub-query in each evaluator on the subset of data generated in that evaluator in the evaluating step above to generate a sub-result;

receive each sub-result from each evaluator at an output handler; and

combine the sub-result from each evaluator into a query result.

20. The non-transitory computer readable medium of claim 19 wherein the query and index database is used to provide for authentication and authorization of requestors submitting queries.