METHODS AND SYSTEMS FOR REAL-TIME CONTINUOUS UPDATES

• US 20090319486A1
• Filed: 06/23/2008
• Published: 12/24/2009
• Est. Priority Date: 06/23/2008
• Status: Active Grant

• Current Assignee: Teradata US, Inc. (Teradata Corporation)
• Original Assignee: Kickfire, Inc. (Teradata Corporation)
• Inventors: Surlaker, Kapil, Chamdani, Joseph I., Meiyyappan, Krishnan, Tran, Hung, Krishnamurthy, Ravi, Branscome, Jeremy, Beck, Alan
• Granted Patent: US 8,458,129 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2329 using versioning
• CPC Class Codes: G06F 16/2477 Temporal data queries

Abstract

Embodiments of the present invention provide fine grain concurrency control for transactions in the presence of database updates. During operations, each transaction is assigned a snapshot version number or SVN. A SVN refers to a historical snapshot of the database that can be created periodically or on demand. Transactions are thus tied to a particular SVN, such as, when the transaction was created. Queries belonging to the transactions can access data that is consistent as of a point in time, for example, corresponding to the latest SVN when the transaction was created. At various times, data from the database stored in a memory can be updated using the snapshot data corresponding to a SVN. When a transaction is committed, a snapshot of the database with a new SVN is created based on the data modified by the transaction and the snapshot is synchronized to the memory. When a transaction query requires data from a version of the database corresponding to a SVN, the data in the memory may be synchronized with the snapshot data corresponding to that SVN.

Claims

1. A method of versioning data in a database, said method comprising:

determining snapshots of data in the database at an interval;

identifying differences in the data;

determining a set of changes that takes the data from one snapshot to another snapshot; and

generating a mask comprising the set of changes that is applied by a machine code database instruction for changing the data from one snapshot to the other snapshot.

2. The method of claim 1, wherein generating the mask comprises generating a mask that is applied by a machine code database instruction for modifying the data in pages of a hardware accelerator memory from one snapshot to another snapshot.

3. The method of claim 1, wherein generating the mask comprises generating a mask having a fixed width format that is independent of the data type being modified.

4. The method of claim 1, wherein generating the mask comprises generating a mask having data that is in a same compressed form as the data in the database.

5. The method of claim 1, further comprising storing a mask for an active snapshot in a host memory and storing at least one additional mask for completed snapshots in the host memory.

6. The method of claim 1, further comprising storing a mask for an active snapshot in a host memory and storing at least one additional mask for completed snapshots in the hardware accelerator.

7. The method of claim 1, further comprising setting the interval based on a configuration.

8. The method of claim 1, further comprising setting the interval based on a size of accumulated changes.

9. The method of claim 1, further comprising applying the mask to data in a hardware accelerator memory upon a transaction commit.

10. The method of claim 1, further comprising applying the mask to data in a hardware accelerator upon demand by a query.

11. A method of enabling queries to continuously operate on different states of data while allowing the data to be manipulated, said method comprising:

receiving queries and transactions on data simultaneously in the database, wherein the queries operate on a data stored in a first memory coupled to a hardware accelerator and the transactions operate on another data stored in a second memory coupled to a host;

taking snapshots of data in the second memory at an interval;

determining a set of changes that takes the data in the first memory from one snapshot to another snapshot;

generating a mask comprising the set of changes that is applied by a machine code database instruction for changing the data in the first memory from one snapshot to the other snapshot.

determining, for each query, a snapshot that is associated with that query; and

determining data in the first memory on which the queries will operate based on the snapshot and selectively applying the mask.

12. The method of claim 11, wherein determining the data on which the queries will operate comprises determining when data on which the queries will operate is absent from the first memory and prefetching the data into the first memory when the data is absent.

13. The method of claim 11, wherein determining the data on which the queries will operate comprises determining when data on which the queries will operate are present in the first memory and changing the data from one snapshot to another snapshot based on the mask.

14. A system that enables queries to operate on different states of data from a database while allowing transactions on the data, wherein the data can be stored in a first memory and a second memory, said system comprising:

a query engine that performs queries on data stored in a first memory;

a transaction engine that performs transactions on at least some of the same data stored in a second memory during the same time that the queries are performed; and

a transaction manager configured to determine snapshots of the data in the second memory at an interval, determine a set of changes that tracks changes in the data in the second memory from one snapshot to another snapshot, and modify data in the first memory on which the queries will operate based on snapshots assigned to each of the queries and selectively requesting a machine code database instruction that applies the changes to the data in the first memory.

15. The system of claim 14, wherein the transaction manager is configured to determine when data on which the queries will operate is absent from the first memory and prefetch the data into the first memory when the data is absent.

16. The system of claim 14, wherein the transaction manager is configured to determine when data on which the queries will operate are present in the first memory and changing the data from one snapshot to another snapshot based on the mask.

17. A computer readable medium comprising program code for performing a method of versioning data in a database, said medium comprising:

program code for determining snapshots of data in the database at an interval;

program code for identifying differences in the data;

program code for determining a set of changes that takes the data from one snapshot to another snapshot; and

program code for generating a mask comprising the set of changes that is applied by a machine code database instruction for changing the data from one snapshot to the other snapshot.

18. The computer readable medium of claim 17, wherein the program code for generating the mask comprises program code for generating a mask that is applied by a machine code database instruction for modifying the data in pages of a hardware accelerator memory from one snapshot to another snapshot.

19. The computer readable medium of claim 17, wherein the program code for generating the mask comprises program code for generating a mask having a fixed width format that is independent of the data type being modified.

20. The computer readable medium of claim 17, wherein the program code for generating the mask comprises program code for generating a mask having data that is in a same compressed form as the data in the database.

21. The computer readable medium of claim 17, further comprising program code for storing a mask for an active snapshot in a host memory and program code for storing at least one additional mask for completed snapshots in the host memory.

22. The computer readable medium of claim 17, further comprising program code for storing a mask for an active snapshot in a host memory and program code for storing at least one additional mask for completed snapshots in the hardware accelerator.

23. The computer readable medium of claim 17, further comprising program code for setting the interval based on a configuration.

24. The computer readable medium of claim 17, further comprising program code for setting the interval based on a size of accumulated changes.

25. The computer readable medium of claim 17, further comprising program code for applying the mask to data in a hardware accelerator memory upon a transaction commit.

26. The computer readable medium of claim 17, further comprising program code for applying the mask to data in a hardware accelerator upon demand by a query.

27. A computer readable medium comprising program code for performing a method of enabling queries to continuously operate on different states of data while allowing the data to be manipulated, said computer readable medium comprising:

program code for receiving queries and transactions on data simultaneously in the database, wherein the queries operate on a data stored in a first memory coupled to a hardware accelerator and the transactions operate on another data stored in a second memory coupled to a host;

program code for taking snapshots of data in the second memory at an interval;

program code for determining a set of changes that takes the data in the first memory from one snapshot to another snapshot;

program code for generating a mask comprising the set of changes that is applied by a machine code database instruction for changing the data in the first memory from one snapshot to the other snapshot.

program code for determining, for each query, a snapshot that is associated with that query; and

program code for determining data on which the queries will operate based on the snapshot and selectively applying the mask.

28. The computer readable medium of claim 27, wherein the program code for determining the data on which the queries will operate comprises program code for determining when data on which the queries will operate is absent from the first memory and program code for prefetching the data into the first memory when the data is absent.

29. The computer readable medium of claim 27, wherein the program code for determining the data on which the queries will operate comprises program code for determining when data on which the queries will operate are present in the first memory and program code for changing the data from one snapshot to another snapshot based on the mask.

30. A method of reconstructing data for a transaction that has been interrupted, said method comprising:

receiving queries and transactions on data simultaneously in the database, wherein the queries operate on data stored in a first memory coupled to a hardware accelerator and the transactions operate on another copy of the data stored in a second memory coupled to a host;

taking snapshots of data in the second memory at an interval;

determining a set of changes that takes the data in the first memory from one snapshot to another snapshot;

detecting an interruption in the transaction;

determining a snapshot in which the interruption occurred;

generating a mask comprising the set of changes that is applied by a machine code database instruction for changing the data from the first memory from one snapshot to the snapshot in which the interruption occurred; and

recovering data from the first memory into the second memory based on the mask.