In-memory row storage architecture

• US 10,664,462 B2
• Filed: 12/29/2017
• Issued: 05/26/2020
• Est. Priority Date: 03/01/2017
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Gurajada, Aditya, Pathak, Amit, Rathod, Paresh, Mittal, Rahul, Sharma, Swati
• Primary Examiner(s): Woo, Isaac M
• Application Number:
  US15/859,175
• Publication Number:
  US 20180253467A1
• Time in Patent Office: 879 Days
• Field of Search: 707600-899
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2212/1044 Space efficiency improvement
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

A system for an in-memory row storage architecture can be provided. In some implementations, the system performs operations comprising determining that data stored at a first location in a persistent page store is active, copying the data to a second location in an in-memory row store in response to determining that the data is active, indexing the data in an index table in response to determining that the data is active, accessing the data at the second location in the in-memory row store based on the index table, and performing an update of the data in the in-memory row store. Related systems, methods, and articles of manufacture are also described.

Claims

1. A system comprising:

at least one processor; and

at least one memory storing instructions which, when executed by the at least one processor, cause operations comprising;

determining, based on one or more accesses of data stored at a first location in a persistent page store, that the data is active;

copying, in response to determining that the data is active, the data to a second location in an in-memory row store;

indexing, in response to determining that the data is active, the data in an index table, the index table comprising indices for rows in the in-memory row store to enable mapping between a virtual page in the in-memory row store and a physical page in the persistent page store; and

accessing, based on the index table, the data at the second location in the in-memory row store to perform an update of the data.

2. The system of claim 1, wherein the operations further comprise:

providing the update of the data within the in-memory row store, wherein the update is not provided within the persistent page store.

3. The system of claim 1, wherein the operations further comprise:

generating, based on a database query, new data; and

inserting the new data into the in-memory row store, wherein the new data is not inserted into the persistent page store until the new data is determined to be inactive.

4. The system of claim 1, wherein the index table comprises indices for each row in the in-memory row store, and wherein each of the indices comprises:

an indication of an identifier for a physical page in the persistent page store;

an indication of a row number in the physical page;

an indication of an identifier for a virtual page in the in-memory row store; and

an indication of a row number in the virtual page.

5. The system of claim 4, wherein the operations further comprise:

processing a database query requiring access to second data; and

determining a third location of a most recent version of the second data based on an index for the second data in the index table, wherein the third location is within the persistent page store or the in-memory row store, and wherein the index provides transparent access for the database query to the second data.

6. The system of claim 1, wherein the data comprises at least one row from a database table having a plurality of rows, wherein a first portion of the plurality of rows is stored in the in-memory row store, and wherein a second portion of the plurality of rows is stored in the persistent page store.

7. The system of claim 1, wherein the operations further comprise:

processing a plurality of transactions comprising a plurality of database statements;

logging, within a transaction log in a persistent disk location, each transaction of the plurality of transactions that affect the in-memory row store when the transaction is committed;

determining that a crash or a power down occurred; and

recovering, after determining that the crash or the power down occurred, at least a portion of the in-memory row store based on accessing the transaction log.

8. The system of claim 7, wherein the operations further comprise:

logging, within the transaction log, changes made by each statement of the plurality of database statements that affect the persistent page store, when the statement is executed;

applying changes made by each statement of the plurality of database statements that affect the in-memory row store to the in-memory row store by creating in-memory row versions, when the statement is executed;

determining, at the end of execution of each statement of the plurality of database statements, whether the statement violates a trigger rule;

processing, when the statement violates the trigger rule, the trigger rule using the in-memory row versions to modify the in-memory row store; and

processing, when the statement violates the trigger rule, the trigger rule using the transaction log to modify the persistent page store.

9. The system of claim 1, wherein the operations further comprise:

determining that an available amount of space in the in-memory row store is insufficient, wherein the amount of space is insufficient when a current transaction cannot complete based on the available amount of space and/or when the available amount of space is less than a threshold;

determining that second data in the in-memory row store is inactive; and

migrating, in response to determining that the available amount of space is too small, the second data from the in-memory row store to the persistent page store.

10. The system of claim 9, wherein determining that the data is active comprises determining that the one or more accesses include the update to the data, and wherein determining that the second data is inactive comprises determining that the second data has not been accessed within a threshold period of time or determining that the second data is a least recently used row in the in-memory row store.

11. A method comprising:

determining, based on one or more accesses of data stored at a first location in a persistent page store, that the data is active;

copying, in response to determining that the data is active, the data to a second location in an in-memory row store;

indexing, in response to determining that the data is active, the data in an index table, the index table comprising indices for rows in the in-memory row store to enable mapping between a virtual page in the in-memory row store and a physical page in the persistent page store; and

accessing, based on the index table, the data at the second location in the in-memory row store to perform an update of the data.

12. The method of claim 11, further comprising:

providing the update of the data within the in-memory row store, wherein the update is not provided within the persistent page store.

13. The method of claim 11, further comprising:

generating, based on a database query, new data; and

inserting the new data into the in-memory row store, wherein the new data is not inserted into the persistent page store until the new data is determined to be inactive.

14. The method of claim 11, wherein the index table comprises indices for each row in the in-memory row store, and wherein each of the indices comprises:

an indication of an identifier for a physical page in the persistent page store;

an indication of a row number in the physical page;

an indication of an identifier for a virtual page in the in-memory row store; and

an indication of a row number in the virtual page.

15. The method of claim 14, further comprising:

processing a database query requiring access to second data; and

determining a third location of a most recent version of the second data based on an index for the second data in the index table, wherein the third location is within the persistent page store or the in-memory row store, and wherein the index provides transparent access for the database query to the second data.

16. A non-transitory computer-readable medium comprising computer program code which, when executed by at least one processor, results in operations comprising:

determining, based on one or more accesses of data stored at a first location in a persistent page store, that the data is active;

copying, in response to determining that the data is active, the data to a second location in an in-memory row store;

indexing, in response to determining that the data is active, the data in an index table, the index table comprising indices for rows in the in-memory row store to enable mapping between a virtual page in the in-memory row store and a physical page in the persistent page store; and

accessing, based on the index table, the data at the second location in the in-memory row store to perform an update of the data.

17. The non-transitory computer readable medium of claim 16, wherein the operations further comprise:

providing the update of the data within the in-memory row store, wherein the update is not provided within the persistent page store.

18. The non-transitory computer readable medium of claim 16, wherein the operations further comprise:

generating, based on a database query, new data; and

inserting the new data into the in-memory row store, wherein the new data is not inserted into the persistent page store until the new data is determined to be inactive.

19. The non-transitory computer readable medium of claim 16, wherein the index table comprises indices for each row in the in-memory row store, and wherein each of the indices comprises:

an indication of an identifier for a physical page in the persistent page store;

an indication of a row number in the physical page;

an indication of an identifier for a virtual page in the in-memory row store; and

an indication of a row number in the virtual page.

20. The non-transitory computer readable medium of claim 19, wherein the operations further comprise:

processing a database query requiring access to second data; and

determining a third location of a most recent version of the second data based on an index for the second data in the index table, wherein the third location is within the persistent page store or the in-memory row store, and wherein the index provides transparent access for the database query to the second data.