Transforming timeseries and non-relational data to relational for complex and analytical query processing
First Claim
1. A method for transforming data to be accessible by a data warehouse, implemented by a computing processor, the method comprising:
- identifying, by the computing processor, a non-relational time series database table comprising time series data, wherein the non-relational time series database table comprises a column containing an array of the time series data collected from at least one source, wherein as new time series data is collected from the at least one source, the new time series data is added to the array instead of adding a new row for the new time series data;
creating, by the computing processor, a virtual view of the time series data in the non-relational time series database table by representing the time series data in the non-relational time series database table as a virtual relational database table, wherein the virtual view is stored as an in-memory storage structure without any intermediate storage, wherein the time series data is stored in the non-relational time series database table and not stored in a relational database table;
receiving a request with a user defined time interval to view the time series data, wherein the request is an SQL request; and
presenting a snapshot of the time series data, in response to the request, by partitioning, by the computing processor, the virtual relational database table into a plurality of virtual partitions according to the user defined time interval across a data time range, wherein each of the plurality of virtual partitions has a respective partition time range that spans the user defined time interval, the plurality of virtual partitions created to optimize loading of the time series data in the virtual view into the snapshot by incrementally refreshing corresponding time series data for a corresponding virtual partition within the data time range.
2 Assignments
0 Petitions
Accused Products
Abstract
A system for transforming time series data into data that is accessible by a data warehouse identifies a data table comprising the time series data. The system creates a virtual view of the data table where the time series data is represented as at least one standard relational table in the virtual view, where the virtual view is presented as a virtual table. The system partitions the virtual table into a plurality of virtual partitions according to a time interval. The virtual table is partitioned across a data time range, where the data time range comprises at least one time interval, and where each of the plurality of virtual partitions has a respective partition time range that spans the time interval. The virtual partitions are created to optimize loading of the data into the data warehouse by incrementally refreshing the data according to the respective partition time range.
-
Citations
23 Claims
-
1. A method for transforming data to be accessible by a data warehouse, implemented by a computing processor, the method comprising:
-
identifying, by the computing processor, a non-relational time series database table comprising time series data, wherein the non-relational time series database table comprises a column containing an array of the time series data collected from at least one source, wherein as new time series data is collected from the at least one source, the new time series data is added to the array instead of adding a new row for the new time series data; creating, by the computing processor, a virtual view of the time series data in the non-relational time series database table by representing the time series data in the non-relational time series database table as a virtual relational database table, wherein the virtual view is stored as an in-memory storage structure without any intermediate storage, wherein the time series data is stored in the non-relational time series database table and not stored in a relational database table; receiving a request with a user defined time interval to view the time series data, wherein the request is an SQL request; and presenting a snapshot of the time series data, in response to the request, by partitioning, by the computing processor, the virtual relational database table into a plurality of virtual partitions according to the user defined time interval across a data time range, wherein each of the plurality of virtual partitions has a respective partition time range that spans the user defined time interval, the plurality of virtual partitions created to optimize loading of the time series data in the virtual view into the snapshot by incrementally refreshing corresponding time series data for a corresponding virtual partition within the data time range. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A computer program product for transforming data to be accessible by a data warehouse, the computer program product comprising:
-
a computer readable storage medium having computer readable program code embodied therewith, the program code executable by a processor to; identify, by the computing processor, a non-relational time series database table comprising time series data, wherein the non-relational time series database table comprises a column containing an array of the time series data collected from at least one source, wherein as new time series data is collected from the at least one source, the new time series data is added to the array instead of adding a new row for the new time series data; create, by the computing processor, a virtual view of the time series data in the non-relational time series database table by representing the time series data in the non-relational time series database table as a virtual relational database table, wherein the virtual view is stored as an in-memory storage structure without any intermediate storage, wherein the time series data is stored in the non-relational time series database table and not stored in a relational database table; receive a request with a user defined time interval to view the time series data, wherein the request is an SQL request; and present a snapshot of the time series data, in response to the request, by partitioning, by the computing processor, the virtual relational database table into a plurality of virtual partitions according to the user defined time interval across a data time range, wherein each of the plurality of virtual partitions has a respective partition time range that spans the user defined time interval, the plurality of virtual partitions created to optimize loading of the time series data in the virtual view into the snapshot by incrementally refreshing corresponding time series data for a corresponding virtual partition within the data time range. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A system comprising:
-
a processor; and a computer readable storage medium operationally coupled to the processor, the computer readable storage medium having computer readable program code embodied therewith to be executed by the processor, the computer readable program code configured to; identify, by the computing processor, a non-relational time series database table comprising time series data, wherein the non-relational time series database table comprises a column containing an array of the time series data collected from at least one source, wherein as new time series data is collected from the at least one source, the new time series data is added to the array instead of adding anew row for the new time series data; create, by the computing processor, a virtual view of the time series data in the non-relational time series database table by representing the time series data in the non-relational time series database table as a virtual relational database table, wherein the virtual view is stored as an in-memory storage structure without any intermediate storage, wherein the time series data is stored in the non-relational time series database table and not stored in a relational database table; receive a request with a user defined time interval to view the time series data, wherein the request is an SQL request; and present a snapshot of the time series data, in response to the request, by partitioning, by the computing processor, the virtual relational database table into a plurality of virtual partitions according to the user defined time interval across a data time range, wherein each of the plurality of virtual partitions has a respective partition time range that spans the user defined time interval, the plurality of virtual partitions created to optimize loading of the time series data in the virtual view into the snapshot by incrementally refreshing corresponding time series data for a corresponding virtual partition within the data time range. - View Dependent Claims (20, 21, 22, 23)
-
Specification