System and methods for optimizing data transfer among various resources in a distributed environment

• US 7,788,243 B2
• Filed: 02/15/2007
• Issued: 08/31/2010
• Est. Priority Date: 09/08/2006
• Status: Active Grant

• Current Assignee: Sybase Incorporated (SAP SE)
• Original Assignee: Sybase Incorporated (SAP SE)
• Inventors: Pasupuleti, Kantikiran Krishna, Nalawade, Vaibhav A., Bhatt, Vadiraja P.
• Primary Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US11/675,323
• Publication Number:
  US 20080065644A1
• Time in Patent Office: 1,293 Days
• Field of Search: 707/8, 707/10, 707/203, 707/2, 707/200, 707/205, 707/625, 707/638, 707/639, 707/655, 707/656, 707/657, 707/687, 707/695, 707/704, 707/737, 711/118, 711/122, 711/129, 711/137, 711/203, 711/100, 711/167, 709/201, 709/203, 709/213, 709/214
• US Class Current: 707/704
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

System providing methodology for optimizing data transfer in a distributed environment is described. In one embodiment, for example, in a distributed shared disk cluster environment comprises a plurality of nodes, each node storing database pages in different size buffer pools configured for local access and configured independent from other nodes in the cluster, the database pages residing at any buffer pool across nodes without requiring that a page be present at an identical buffer pool at all nodes in the cluster, a method of the present invention for transferring database pages among nodes comprises: generating a request at a first node for a set of database pages, the request including a timestamp for determining what are latest versions of the database pages; in response to the request, determining where latest versions of the database pages of the set reside among the plurality of nodes without regard to what size of buffer pool any given database page may reside in; if the database pages of the set reside among the plurality of nodes, initiating transfer of particular database pages to the first node as required for providing the first node with the latest versions of the database pages; and, otherwise if the database pages of the set do not reside among the plurality of nodes, instructing the first node to fetch the database pages from disk.

Claims

1. In a distributed shared disk cluster environment comprising a plurality of nodes, each node storing database pages in different size buffer pools configured for local access and configured independent from other nodes in the cluster, the database pages residing at any buffer pool across nodes without requiring that a page be present at an identical buffer pool at all nodes in the cluster, a method for transferring database pages among nodes comprising:

generating a request at a first node for a set of database pages, the request including a timestamp for determining by a lock manager what are latest versions of the database pages, wherein the lock manager centrally stores timestamp information from all nodes of the cluster;

in response to the request, determining by said lock manager where latest versions of the database pages of the set reside among the plurality of nodes without regard to what size of buffer pool any given database page may reside in, wherein determining by said lock manager includes evaluating the timestamp received from the first node to determine if the first node has the latest versions of the database pages;

if the database pages of the set reside among the plurality of nodes, initiating transfer of particular database pages to the first node as required for providing the first node with the latest versions of the database pages; and

, otherwise

if the database pages of the set do not reside among the plurality of nodes, instructing the first node to fetch the database pages from disk.

2. The method of claim 1, wherein said database pages comprise database index pages.

3. The method of claim 1, wherein said database pages comprise database table pages.

4. The method of claim 1, wherein said request comprises a request for a contiguous block of database pages.

5. The method of claim 1, wherein said request comprises a starting page number and a number of pages, for specifying a set of database pages.

6. The method of claim 1, further comprising:

in response to the request, selecting a buffer pool at the first node for receiving the set of database pages.

7. The method of claim 6, wherein said initiating transfer step includes:

transferring a direct memory access (DMA) key, so that database pages may be transferred directly to the selected buffer pool at the first node.

8. The method of claim 1, further comprising:

in response to the request, determining whether some of the database pages of the set already reside at the first node.

9. The method of claim 8, further comprising:

if the latest version of one or more of the database pages of the set already reside at the first node, downgrading the request so that the request no longer includes database pages residing at the first node.

10. The method of claim 1, wherein said request includes a lock request for the set of database pages.

11. The method of claim 1, wherein said determining step includes:

based on the timestamp, determining by a lock manager which database pages of the set are required to be transferred to the first node, in order to satisfy the request.

12. The method of claim 11, wherein the lock manager specifies that a database page should be transferred from a given remote node, when a corresponding timestamp for the database page at the given remote node has advanced relative to the timestamp of the first node.

13. The method of claim 1, wherein a lock manager instructs the first node to fetch the database pages from disk in a single read operation, when the database pages of the set do not reside among the plurality of nodes.

14. A computer-readable medium having processor-executable instructions for performing the method of claim 1.

15. A distributed shared disk cluster system comprising:

a plurality of nodes in the cluster, each node storing database pages in different size buffer pools configured for local access and configured independent from other nodes in the cluster, the database pages residing at any buffer pool across nodes without requiring that a page be present at an identical buffer pool at all nodes in the cluster;

a first node in the cluster that generates a request for a set of database pages, the request including a timestamp for determining what are latest versions of the database pages; and

a lock manager centrally storing timestamp information from all nodes of the cluster that, responsive to said request, that determines where latest versions of the database pages of the set reside among the plurality of nodes including evaluating the timestamp received from the first node to determine if the first node has the latest versions of the database pages, without regard to what size of buffer pool any given database page may reside, and which initiates transfer of particular database pages to the first node as follows;

if the database pages of the set reside among the plurality of nodes, initiating transfer of particular database pages to the first node as required for providing the first node with the latest versions of the database pages; and

, otherwise

if the database pages of the set do not reside among the plurality of nodes, instructing the first node to fetch the database pages from disk.

16. The system of claim 15, wherein said database pages comprise database index pages.

17. The system of claim 15, wherein said database pages comprise database table pages.

18. The system of claim 15, wherein said request comprises a request for a contiguous block of database pages.

19. The system of claim 15, wherein said request comprises a starting page number and a number of pages, for specifying a set of database pages.

20. The system of claim 15, wherein the first node selects a local buffer pool at the first node for receiving the set of database pages.

21. The system of claim 20, wherein database pages may be transferred directly to the selected buffer pool at the first node using a direct memory access (DMA) key.

22. The system of claim 15, further comprising:

program logic for determining whether some of the database pages of the set already reside at the first node.

23. The system of claim 22, wherein the lock manager downgrades the request when the latest version of one or more of the database pages already resides at the first node.

24. The system of claim 15, wherein said request includes a lock request for the set of database pages.

25. The system of claim 15, wherein the lock manager determines based on the timestamp which database pages of the set are required to be transferred to the first node, in order to satisfy the request.

26. The system of claim 25, wherein the lock manager specifies that a database page should be transferred from a given remote node, when a corresponding timestamp for the database page at the given remote node has advanced relative to the timestamp of the first node.

27. The system of claim 15, wherein the lock manager instructs the first node to fetch the database pages from disk in a single read operation, when the database pages of the set do not already reside among the plurality of nodes.

28. The system of claim 15, wherein database pages are transferred to the first node asynchronously.

29. In a distributed database environment comprising a database system utilizing a shared disk cluster, an improved method for transferring database information, the method comprising:

storing the database information among a plurality of nodes in the cluster, each node storing particular database information as database pages in different size buffer pools configured for local access and configured independent from other nodes in the cluster;

receiving a request from a requesting node for a set of database pages, the request including a timestamp for determining by a lock manager what are latest versions of the database pages, wherein the lock manager centrally stores timestamp information from all nodes in the cluster;

in response to the request, determining by said lock manager which nodes own latest versions of the database pages of the set among the plurality of nodes without regard to what size of buffer pool any given database page may reside, wherein determining by said lock manager includes evaluating the timestamp received from the requesting node to determine if the requesting node has the latest versions of the database pages;

if all of the database pages of the set are owned by nodes, transferring to the requesting node any database pages required for providing the requesting node with the latest versions; and

, otherwise

if the database pages of the set are not owned by the nodes, instructing the requesting node to fetch the database pages from disk.

30. The method of claim 29, wherein said database pages comprise database index pages.

31. The method of claim 29, wherein said database pages comprise database table pages.

32. The method of claim 29, wherein said request comprises a request for a contiguous block of database pages.

33. The method of claim 29, wherein said request comprises a starting page number and a number of pages, for specifying a set of database pages.

34. The method of claim 29, further comprising:

in response to the request, selecting a buffer pool at the requesting node for receiving the set of database pages.

35. The method of claim 34, wherein said transferring step includes:

transferring a direct memory access (DMA) key, so that database pages may be transferred directly to the selected buffer pool at the requesting node.

36. The method of claim 29, further comprising:

in response to the request, determining whether some of the database pages of the set already reside at the requesting node.

37. The method of claim 36, further comprising:

resizing the request according to the availability of the buffer pools at the requesting node, and if the latest version of one or more of the database pages of the set already reside at the requesting node, resizing the request so that the request no longer includes database pages residing at the requesting node.

38. The method of claim 29, wherein said request includes a lock request for the set of database pages.

39. The method of claim 29, wherein said determining step includes:

based on the timestamp, determining by a lock manager which database pages of the set are required to be transferred to the requesting node, in order to satisfy the request.

40. The method of claim 39, wherein the lock manager specifies that a

particular database page should be transferred from a given node that owns the particular database page, when a corresponding timestamp for the particular database page at the given node has advanced relative to the timestamp of the requesting node.

41. The method of claim 29, wherein a lock manager instructs the requesting node to fetch the database pages from disk in a single read operation, when the database pages of the set do not reside among the plurality of nodes.

42. The method of claim 29, further comprising:

storing on a computer-readable medium processor-executable instructions for performing the method of claim 29.