CHECKPOINTS FOR DOCUMENT STORE

• US 20190324866A1
• Filed: 04/23/2018
• Published: 10/24/2019
• Est. Priority Date: 04/23/2018
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Bensberg, Christian, Geissinger, Steffen, Glebe, Thorsten
• Granted Patent: US 10,810,092 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1474 in transactions G06F16/20 takes precedence
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Provided are systems and methods for storing checkpoints of an in-memory storage device. In one example, the method may include detecting database transactions performed on a slice of an in-memory storage and storing a record of the database transactions in a log file, receiving a request to persist the slice of in-memory storage, the request comprising a timing indicator corresponding to a timing of database transactions, trimming the log file based on the timing indicator to remove a record of one or more database transactions from the log file that occurred prior to the timing indicator and to preserve a record of one or more database transactions that occurred after the timing indicator, and persisting, to disk, data from the slice of in-memory storage and the trimmed log file preserving the record of the one or more database transactions that occurred after the timing indicator.

Claims

1. A computing system comprising:

an in-memory storage; and

a processor configured to

detect database transactions performed on a slice of the in-memory storage and store a record of the database transactions in a log file,

receive a request to persist the slice of in-memory storage, the request comprising a timing indicator corresponding to a timing of database transactions,

trim the log file based on the timing indicator to remove a record of one or more database transactions from the log file that occurred prior to the timing indicator and to preserve a record of one or more database transactions that occurred after the timing indicator, and

persist, to disk, data present in the slice based on a point in time of the timing indicator and the trimmed log file preserving the record of the one or more database transactions that occurred after the timing indicator.

2. The computing system of claim 1, wherein the in-memory storage comprise a document store storing database records in a document model format.

3. The computing system of claim 1, wherein the processor is further configured to remove a database result of the one or more database transactions that occurred after the timing indicator from the slice, prior to the slice being persisted to disk.

4. The computing system of claim 1, wherein the timing indicator comprises a counter that represents a time at which a most recently committed database transaction was performed on the slice and that is visible to all database transactions.

5. The computing system of claim 4, wherein the processor is configure to remove a record of all database transactions from the log file that have a timestamp less than or equal to the counter and preserve a record of all database transactions in the log file having a timestamp greater than the counter.

6. The computing system of claim 1, wherein the processor is further configured to insert an identification of the timing indicator within metadata of the trimmed log file prior to the trimmed log file being persisted to disk.

7. The computing system of claim 1, wherein the processor is further configured to receive a request to reload the persisted slice from disk, and in response, load the persisted slice and the trimmed log file, and execute the database transactions stored in the trimmed log file to reconstruct a current state of the slice in the in-memory.

8. The computing system of claim 1, wherein the processor is further configured to lock the slice from write access such that database transactions are prevented from writing to the slice while the trimming of the log file and the persisting of the slice are performed.

9. A method comprising:

detecting database transactions performed on a slice of an in-memory storage and storing a record of the database transactions in a log file;

receiving a request to persist the slice of in-memory storage, the request comprising a timing indicator corresponding to a timing of database transactions;

trimming the log file based on the timing indicator to remove a record of one or more database transactions from the log file that occurred prior to the timing indicator and to preserve a record of one or more database transactions that occurred after the timing indicator; and

persisting, to disk, data present in the slice based on a point in time of the timing indicator and the trimmed log file preserving the record of the one or more database transactions that occurred after the timing indicator.

10. The method of claim 9, wherein the in-memory storage comprise a document store storing database records in a document model format.

11. The method of claim 9, further comprising removing a database result of the one or more database transactions that occurred after the timing indicator from the slice, prior to persisting the slice to disk.

12. The method of claim 9, wherein the timing indicator comprises a counter that represents a time at which a most recently committed database transaction was performed on the slice and that is visible to all database transactions.

13. The method of claim 12, wherein the trimming comprises removing a record of all database transactions from the log file that have a timestamp less than or equal to the counter and preserving a record of all database transactions in the log file having a timestamp greater than the counter.

14. The method of claim 9, further comprising inserting an identification of the timing indicator within metadata of the trimmed log file prior to persisting the trimmed log file to disk.

15. The method of claim 9, further comprising receiving a request to reload the persisted slice from disk, and in response, loading the persisted slice and the trimmed log file, and executing the database transactions included in the trimmed log file to reconstruct a current state of the slice in the in-memory.

16. The method of claim 9, further comprising locking the slice from write access such that database transactions are prevented from writing to the slice while the trimming of the log file and the persisting of the slice are performed.

17. A non-transitory computer-readable storage medium storing program instructions that when executed cause a processor to perform a method comprising:

detecting database transactions performed on a slice of an in-memory storage and storing a record of the database transactions in a log file;

receiving a request to persist the slice of in-memory storage, the request comprising a timing indicator corresponding to a timing of database transactions;

trimming the log file based on the timing indicator to remove a record of one or more database transactions from the log file that occurred prior to the timing indicator and to preserve a record of one or more database transactions that occurred after the timing indicator; and

persisting, to disk, data present in the slice based on a point in time of the timing indicator and the trimmed log file preserving the record of the one or more database transactions that occurred after the timing indicator.

18. The non-transitory computer readable medium of claim 17, wherein the in-memory storage comprise a document store storing database records in a document model format.

19. The non-transitory computer readable medium of claim 17, wherein the method further comprises removing a database result of the one or more database transactions that occurred after the timing indicator from the slice, prior to persisting the slice to disk.

20. The non-transitory computer readable medium of claim 17, wherein the timing indicator comprises a counter that represents a time at which a most recently committed database transaction was performed on the slice and that is visible to all database transactions.