Distributed database system providing data and space management methodology

• US 8,266,111 B2
• Filed: 03/26/2008
• Issued: 09/11/2012
• Est. Priority Date: 11/01/2004
• Status: Active Grant

• Current Assignee: Sybase Incorporated (SAP SE)
• Original Assignee: Sybase Incorporated (SAP SE)
• Inventors: Lin, Mei-Lin Linda, Zhou, Fei, Francis, Joe, Sampath, Srikanth, Ramachandran, Satya N., Varakur, Gangavara Prasad
• Primary Examiner(s): Leroux, Etienne
• Assistant Examiner(s): Jacob, Ajith
• Application Number:
  US12/056,201
• Publication Number:
  US 20080235298A1
• Time in Patent Office: 1,630 Days
• Field of Search: 707/674
• US Class Current: 707/674
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database

Abstract

A distributed database system providing data and space management methodology. In a distributed database system having an allocation page for tracking allocation and deallocation of data pages including a deallocation bit for tracking data page deallocation, a method for determining when to clear the deallocation bit during recovery comprises: during runtime operation, generating log records recording allocation and deallocation of data pages, setting the deallocation bit during deallocation of a data page by a transaction, and clearing the deallocation bit after the transaction has committed; during recovery following a failure, identifying a particular log record corresponding to the deallocation bit set during deallocation of a given data page; determining whether the deallocation of the given data page recorded in the particular log record committed prior to the failure; and if the deallocation of the given data page committed prior to the failure, clearing the deallocation bit.

Claims

1. in a distributed database system comprising a cluster of nodes and having an allocation page for tracking allocation and deallocation of data pages accessed by multiple nodes, the allocation page including a deallocation bit for tracking deallocation of a data page, a method for determining when to clear the deallocation bit during recovery from failure of a node while other nodes of the distributed database system remains online, the method comprising:

during runtime operation of the distributed database system, generating log records recording allocation and deallocation of data pages required by the cluster of nodes, setting the deallocation bit during deallocation of a data page by a transaction, and clearing the deallocation bit after the transaction has committed;

during recovery of the distributed database system following a failure of a given node and while other nodes of the distributed database system remains online, determining ownership of the deallocation bit by identifying a particular log record corresponding to the deallocation bit set in the allocation page during deallocation of a given data page;

determining whether the deallocation of the given data page recorded in the particular log record committed prior to the failure; and

if the deallocation of the given data page committed prior to the failure and the ownership of the deallocation bit can be established to the particular log record, clearing the deallocation bit.

2. The method of claim 1, further comprising:

if the deallocation was not committed prior to the failure, redoing the deallocation and any necessary undo action without clearing the deallocation bit.

3. The method of claim 1, wherein said step of generating log records includes generating log records of deallocation of data pages without generating log records for post-commit work performed after the transaction has committed.

4. The method of claim 1, wherein the failure comprises a complete failure or a partial failure of the distributed database system.

5. The method of claim 1, wherein the failure includes a partial failure of the distributed database system after which one or more nodes of the distributed database system continue runtime operations.

6. The method of claim 5, wherein said identifying step includes identifying whether the deallocation bit is set as a result of a transaction committed prior to the partial failure of the distributed database system which has post-commit work pending.

7. The method of claim 1, wherein said clearing step includes substeps of:

determining whether the deallocation bit in the allocation page is set;

if the deallocation bit in the allocation page is set, identifying a particular log record corresponding to the deallocation bit;

comparing a data page timestamp to a new allocation page timestamp recorded in the particular log record; and

if the data page timestamp is less than the new allocation page timestamp recorded in the particular log record, clearing the deallocation bit.

8. The method of claim 1, wherein said determining step includes consulting a lock manager for determining state of the data page, so as to minimize page transfers and page reads from disk in the distributed database system.

9. A computer-readable medium having processor-executable instructions for performing the method of claim 1.

10. A downloadable set of processor-executable instructions for performing the method of claim 1.

11. A system for tracking allocation and deallocation of database pages in a distributed database system having a plurality of database servers sharing access to data pages, the system comprising:

an allocation page for tracking allocation and deallocation of data pages in the database system, the allocation page including a deallocation bit for tracking deallocation of a data page, an allocation module for setting the deallocation bit during deallocation of a data page by a transaction, and clearing the deallocation bit after the transaction has committed during runtime operation of the distributed database system;

a log module for generating log records recording allocation and deallocation of data pages during runtime operation; and

a recovery module for identifying, during recovery from failure of a given database server and while other database servers of the distributed database system remains online, ownership of the deallocation bit by determining a particular log record corresponding to the deallocation bit set in the allocation page during deallocation of a given data page, clearing the deallocation bit if the deallocation of the given data page recorded in the particular log record committed prior to the failure and the ownership of the deallocation bit can be established to the particular log record, and redoing the deallocation and performing any necessary undo action if the deallocation was not committed prior to failure.

12. The system of claim 11, wherein the recovery module redoes the deallocation and any necessary undo action without clearing the deallocation bit if the deallocation was not committed prior to failure.

13. The system of claim 11, wherein the log module generates log records of deallocation of data pages without generating log records for post-commit work performed after the transaction has committed.

14. The system of claim 11, wherein the failure comprises a failure of more than one database server of the distributed database system.

15. The system of claim 11, wherein the failure comprises a failure of one database server node of the distributed database system after which one or more other nodes of the distributed database system continue runtime operations.

16. The system of claim 15, wherein the recovery module determines whether the deallocation bit is set as a result of a transaction committed prior to the failure which has post-commit work pending.

17. The system of claim 11, wherein the recovery module determines whether the deallocation bit in the allocation page is set and identifies a particular log record corresponding to the deallocation bit if the deallocation bit in the allocation page is set.

18. The system of claim 17, wherein the recovery module compares a data page timestamp to a new allocation page timestamp recorded in the particular log record for purposes of determining whether post-commit processing is complete.

19. The system of claim 11, wherein the recovery module consults a lock manager for determining state of the data page, so as to minimize page transfers and page reads from disk in the distributed database system.

20. In a distributed database system including a plurality of servers sharing access to data, an improved method for post-commit processing of transactions, the method comprising:

maintaining an allocation page timestamp on an allocation page which allocation page timestamp is updated when pages are deallocated;

when data pages are deallocated during operation of the distributed database system, setting a deallocation bit on the allocation page;

creating log records identifying the deallocated data pages and the allocation page timestamp before and after the deallocation;

during database recovery operations following a failure of a given server and while other servers of the distributed database system remain online, determining whether to perform post-commit processing for a committed transaction based on examining the log records to determine if the data page deallocation committed and whether ownership of the deallocation bit can be established to a given log record as a result of the data page deallocation being reflected in that given log record; and

performing any post-commit processing determined to be necessary to return the database to a consistent state.

21. The method of claim 20, further comprising:

if said determining step determines that the data page deallocation was not committed prior to the failure, redoing the deallocation and any necessary undo action without clearing the deallocation bit.

22. The method of claim 20, wherein said step of creating log records includes creating log records of deallocation of data pages without generating log records for post-commit work performed after the transaction has committed.

23. The method of claim 20, wherein the failure comprises a complete failure or a partial failure of the distributed database system.

24. The method of claim 20, wherein the failure includes a partial failure of the distributed database system after which one or more servers continue runtime operations.

25. The method of claim 24, wherein said determining step includes identifying whether the deallocation bit is set as a result of a transaction committed prior to the partial failure of the distributed database system which has post-commit work pending.

26. The method of claim 25, further comprising:

if the data page deallocation committed prior to the failure, clearing the deallocation bit.

27. The method of claim 20, wherein said determining step includes substeps of:

determining whether the deallocation bit in the allocation page is set;

if the deallocation bit in the allocation page is set, identifying a particular log record corresponding to the deallocation bit; and

comparing a data page timestamp to an allocation page timestamp recorded in the particular log record.

28. The method of claim 20, wherein said determining step includes consulting a lock manager for determining state of the data page.

29. The method of claim 20, wherein said step of performing post commit work includes performing post-commit work for a transaction committed prior to the failure which is determined to have post-commit work pending.

30. A computer-readable medium having processor-executable instructions for performing the method of claim 20.