Transforming time series data points from concurrent processes

• US 10,282,361 B2
• Filed: 04/29/2016
• Issued: 05/07/2019
• Est. Priority Date: 04/29/2016
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Subramanian, Anand, Valine, Thomas Nicholas, Sura, Bhinav, Sarkapally, Rajavardhan
• Primary Examiner(s): Pham, Khanh B
• Application Number:
  US15/142,441
• Publication Number:
  US 20170316067A1
• Time in Patent Office: 1,103 Days
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

Transforming time series data points from concurrent processes is described. A time series database system writes, to a queue, a first time series data point received from a first process. The time series database system writes, to the queue, a second time series data point received from a second process that executes concurrently with the first process. The time series database system removes the first time series data point and the second time series data point from the queue. The time series database system creates transformed time series data by applying a transformation to the first time series data point and the second time series data point. The time series database system outputs the transformed time series data to a user device.

Claims

1. A system for a transforming time series data points from concurrent processes, the system comprising:

one or more processors; and

a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to;

write to a queue a first time series data point received from a first process associated with a first computer amongst a cluster of computers;

write to the queue a second time series data point received from a second process associated with a second computer amongst the cluster of computers, the second process executing concurrently with the first process;

move the first time series data point and the second time series data point from the queue to a first data buffer of a non-persistent storage, the first time series data point and the second time series data point moved in response to determining the queue has stored a predetermined amount of time series data from the cluster of computers for a time period corresponding to the first time series data point and the second time series data point;

create transformed time series data by applying a transformation to the moved first time series data point and the moved second time series data point of the first data buffer, and writing the transformed time series data to a second data buffer of the non-persistent storage;

write the transformed time series data, the first time series data point, and the second time series data point to a persistent storage after applying the transformation to create the transformed time series data; and

output the transformed time series data to a user device.

2. The system of claim 1, wherein writing the first time series data point to the queue comprises protecting the queue from other access, writing the first time series data point to the protected access queue, and enabling other access to the protected access queue;

and writing the second time series data point to the queue comprises protecting the queue from other access, writing the second time series data point to the protected access queue, and enabling other access to the protected access queue.

3. The system of claim 1, wherein moving the first time series data point and the second time series data point from the queue comprises protecting the queue from other access, removing the first time series data point and the second time series data point from the protected access queue, and enabling other access to the protected access queue.

4. The system of claim 1, wherein the first data buffer of the non-persistent storage comprises a raw data buffer.

5. The system of claim 4, wherein creating the transformed time series data by applying the transformation to the first time series data point and the second time series data point comprises applying the transformation to a copy of the first time series data point from the raw data buffer and a copy of the second time series data point from the raw data buffer, and wherein the second data buffer of the non-persistent storage comprises a transformed data buffer.

6. The system of claim 1, comprising further instructions, which when executed, cause the one or more processors to:

create additional transformed time series data by applying an additional transformation to one of the transformed time series data and a set of the first time series data point and the second time series data point;

write the additional transformed time series data to data storage; and

output the additional transformed time series data to the user device.

7. A computer program product comprising computer-readable program code to be executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code including instructions to:

write to a queue, by a time series database system, a first time series data point received from a first process associated with a first computer amongst a cluster of computers;

write to the queue, by the time series database system, a second time series data point received from a second process associated with a second computer amongst the cluster of computers, the second process executing concurrently with the first process;

move, by the time series database system, the first time series data point and the second time series data point from the queue to a first data buffer of a non-persistent storage, the first time series data point and the second time series data point moved in response to determining the queue has stored a predetermined amount of time series data from the cluster of computers for a time period corresponding to the first time series data point and the second time series data point;

create, by the time series database system, transformed time series data by applying a transformation to the moved first time series data point and the moved second time series data point of the first data buffer, and

write the transformed time series data to a second data buffer of the non-persistent storage;

write the transformed time series data, the first time series data point, and the second time series data point to a persistent storage after applying the transformation to create the transformed time series data; and

output, by the time series database system, the transformed time series data to a user device.

8. The computer program product of claim 7, wherein writing the first time series data point to the queue comprises protecting the queue from other access, writing the first time series data point to the protected access queue, and enabling other access to the protected access queue;

and writing the second time series data point to the queue comprises protecting the queue from other access, writing the second time series data point to the protected access queue, and enabling other access to the protected access queue.

9. The computer program product of claim 7, wherein moving the first time series data point and the second time series data point from the queue comprises protecting the queue from other access, removing the first time series data point and the second time series data point from the protected access queue, and enabling other access to the protected access queue.

10. The computer program product of claim 7, wherein the first data buffer of the non-persistent storage comprises a raw data buffer.

11. The computer program product of claim 10, wherein creating the transformed time series data by applying the transformation to the first time series data point and the second time series data point comprises applying the transformation to a copy of the first time series data point from the raw data buffer and a copy of the second time series data point from the raw data buffer, and wherein the second data buffer of the non-persistent storage comprises a transformed data buffer.

12. The computer program product of claim 7, wherein the program code comprises further instructions to:

create, by the time series database system, additional transformed time series data by applying an additional transformation to one of the transformed time series data and a set of the first time series data point and the second time series data point;

write, by the time series database system, the additional transformed time series data to data storage; and

output, by the time series database system, the additional transformed time series data to the user device.

13. A method for transforming time series data points from concurrent processes, the method comprising:

writing to a queue, by a time series database system, a first time series data point received from a first process associated with a first computer amongst a cluster of computers;

writing to the queue, by the time series database system, a second time series data point received from a second process associated with a second computer amongst the cluster of computers, the second process executing concurrently with the first process;

moving, by the time series database system, the first time series data point and the second time series data point from the queue to a first data buffer of a non-persistent storage, the first time series data point and the second time series data point moved in response to determining the queue has stored a predetermined amount of time series data from the cluster of computers for a time period corresponding to the first time series data point and the second time series data point;

creating, by the time series database system, transformed time series data by applying a transformation to the moved first time series data point and the moved second time series data point of the first data buffer, and

writing the transformed time series data to a second data buffer of the non-persistent storage;

writing the transformed time series data, the first time series data point, and the second time series data point to a persistent storage after applying the transformation to create the transformed time series data; and

outputting, by the time series database system, the transformed time series data to a user device.

14. The method of claim 13, wherein writing the first time series data point to the queue comprises protecting the queue from other access, writing the first time series data point to the protected access queue, and enabling other access to the protected access queue;

and writing the second time series data point to the queue comprises protecting the queue from other access, writing the second time series data point to the protected access queue, and enabling other access to the protected access queue.

15. The method of claim 13, wherein moving the first time series data point and the second time series data point from the queue comprises protecting the queue from other access, removing the first time series data point and the second time series data point from the protected access queue, and enabling other access to the protected access queue;

and wherein the first data buffer of the non-persistent storage comprises a raw data buffer.

16. The method of claim 15, wherein creating the transformed time series data by applying the transformation to the first time series data point and the second time series data point comprises applying the transformation to a copy of the first time series data point from the raw data buffer and a copy of the second time series data point from the raw data buffer, and wherein the second data buffer of the non-persistent storage comprises a transformed data buffer.

17. The method of claim 13, wherein the method further comprises:

creating, by the time series database system, additional transformed time series data by applying an additional transformation to one of the transformed time series data and a set of the first time series data point and the second time series data point;

writing, by the time series database system, the additional transformed time series data to data storage; and

outputting, by the time series database system, the additional transformed time series data to the user device.