Method and system for performing a redistribute transparently in a multi-node system

• US 20070162506A1
• Filed: 01/12/2006
• Published: 07/12/2007
• Est. Priority Date: 01/12/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Grosman, Ronen, Romanufa, Keriley, Van Boeschoten, Robin
• Application Number:
  US11/330,554
• Publication Number:
  US 20070162506A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

A method for performing a redistribute of data in a database system including a plurality of nodes is disclosed. The data includes a plurality of partitions distributed between the plurality of nodes. At least one new node is being added. The method and system include selecting at least one partition of the plurality of partitions to be moved from the plurality of nodes only to the at least one new node. The method and system also include moving the at least one partition only to the at least one new node. The method and system further include removing the at least one partition from the plurality of nodes.

Claims

1. A method for performing a redistribute of data in a database system including a plurality of nodes, the data including a plurality of partitions distributed between the plurality of nodes, at least one new node being added, the method comprising:

selecting at least one partition of the plurality of partitions to be moved from the plurality of nodes only to the at least one new node;

moving the at least one partition only to the at least one new node; and

removing the at least one partition from the plurality of nodes.

2. The method of claim 1 wherein each of the plurality of nodes and each of the at least one new node include a portion of the data, the selecting further including:

choosing the at least one partition to minimize a difference between the portion of the data in each of the plurality of nodes and each of the at least one new node.

3. The method of claim 2 wherein the portion of the data for each of the plurality of nodes and each of the at least one new node corresponds to a weight and wherein the choosing further includes:

selecting the at least one partition such a weight difference that the weight for each of the plurality of nodes and each of the at least new node is minimized.

4. The method of claim 1 wherein the database system includes at least one multidimensional clustering (MDC) table, the at least one MDC table determining the plurality of partitions.

5. The method of claim 1 wherein the database system is a shared disk environment.

6. The method of claim 1 wherein the database system is not a shared disk environment and wherein the moving further includes:

shipping at least one partition across at least one disk.

7. The method of claim 1 wherein the database system includes a shared file system.

8. The method of claim 1 wherein database system does not includes a shared file system and wherein the moving further includes:

shipping at least one file container for the at least one partition.

9. The method of claim 1 wherein each of the at least one partition corresponds to an index and wherein the moving further includes:

transparently accounting for the index.

10. The method of claim 9 wherein the transparently accounting further includes:

providing a new index for each of the at least one partition.

11. The method of claim 9 wherein the transparently accounting includes:

marking the index for each of the at least one partition as deleted.

12. The method of claim 1 wherein each of the at least one partition corresponds to an index and wherein the moving further includes:

copying the at least one partition to the at least one node;

building the new index on each of the at least one node for each of the at least one partition;

maintaining an activity log for each of the at least one partition on the plurality of nodes;

suspending access to the data;

applying the activity log for each of the at least one partition on each of the at least one node; and

marking the index for each of the at least one partition as deleted.

13. The method of claim 1 wherein each of the at least one partition corresponds to an index, wherein the database system includes a memory and at least one disk and wherein the moving further includes:

storing any update to the at least one partition in memory;

maintaining an activity log for each of the at least one partition on the plurality of nodes;

building the new index on each of the at least one node for each of the at least one partition;

applying the activity log for each of the at least one partition on each of the at least one node;

copying the at least one partition to the at least one new node;

suspending access to the at least one partition;

reapplying the activity log for each of the at least one partition on each of the at least one node.

14. A system for performing a redistribute of data in a database system including a plurality of nodes, the data including a plurality of partitions distributed between the plurality of nodes, at least one new node being added, the method comprising:

an element for selecting at least one partition of the plurality of partitions to be moved from the plurality of nodes only to the at least one new node;

an element for moving the at least one partition only to the at least one new node; and

an element for removing the at least one partition from the plurality of nodes.

15. A computer program product comprising a computer-readable medium including a program for performing a redistribute of data in a database system including a plurality of nodes, the data including a plurality of partitions distributed between the plurality of nodes, at least one new node being added, the program including instructions for:

selecting at least one partition of the plurality of partitions to be moved from the plurality of nodes only to the at least one new node;

moving the at least one partition only to the at least one new node; and

removing the at least one partition from the plurality of nodes.

16. The computer program product comprising a of claim 15 wherein each of the plurality of nodes and each of the at least one new node include a portion of the data, and wherein the selecting instructions further include instructions for:

choosing the at least one partition to minimize a difference between the portion of the data in each of the plurality of nodes and each of the at least one new node.

17. The computer program product comprising a of claim 16 wherein the portion of the data for each of the plurality of nodes and each of the at least one new node corresponds to a weight and wherein the choosing instructions further include instructions fir:

selecting the at least one partition such a weight difference that the weight for each of the plurality of nodes and each of the at least new node is minimized.

18. The computer program product comprising a of claim 15 wherein the database system includes at least one multidimensional clustering (MDC) table, the at least one MDC table determining the plurality of partitions.

19. The computer program product comprising a of claim 15 wherein each of the at least one partition corresponds to an index and wherein the moving further instructions includes instructions for:

transparently accounting for the index.

20. The computer program product comprising a of claim 15 wherein each of the at least one partition corresponds to an index and wherein the moving further instructions include instructions for:

copying the at least one partition to the at least one node;

building the new index on each of the at least one node for each of the at least one partition;

maintaining an activity log for each of the at least one partition on the plurality of nodes;

suspending access to the data;

applying the activity log for each of the at least one partition on each of the at least one node; and

marking the index for each of the at least one partition as deleted.

21. The computer program product comprising a of claim 15 wherein each of the at least one partition corresponds to an index, wherein the database system includes a memory and at least one disk and wherein the moving further includes:

storing any update to the at least one partition in memory;

maintaining an activity log for each of the at least one partition on the plurality of nodes;

building the new index on each of the at least one node for each of the at least one partition;

applying the activity log for each of the at least one partition on each of the at least one node;

copying the at least one partition to the at least one new node;

suspending access to the at least one partition;

reapplying the activity log for each of the at least one partition on each of the at least one node.