Highly scalable and highly available cluster system management scheme

• US 20020091814A1
• Filed: 01/07/2002
• Published: 07/11/2002
• Est. Priority Date: 07/10/1998
• Status: Active Grant

• Current Assignee: BlueCat Networks Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Mancisidor, Rodolfo Ausgusto, Arendt, James W., Chao, Ching-Yun
• Granted Patent: US 7,412,479 B2
• Time in Patent Office: Days
• US Class Current: 709/223
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources

Abstract

A cluster system is treated as a set of resource groups, each resource group including an highly available application and the resources upon which it depends. A resource group may have between 2 and M data processing systems, where M is small relative to the cluster size N of the total cluster. Configuration and status information for the resource group is fully replicated only on those data processing systems which are members of the resource group. A configuration object/database record for the resource group has an associated owner list identifying the data processing systems which are members of the resource group and which may therefore manage the application. A data processing system may belong to more than one resource group, however, and configuration and status information for the data processing system is replicated to each data processing system which could be affected by failure of the subject data processing system—that is, any data processing system which belongs to at least one resource group also containing the subject data processing system. The partial replication scheme of the present invention allows resource groups to run in parallel, reduces the cost of data replication and access, is highly scalable and applicable to very large clusters, and provides better performance after a catastrophe such as a network partition.

Claims

8. A method of managing cluster configuration information, comprising:

dividing a cluster into a plurality of resource groups each including an application and at least two data processing systems;

instantiating a plurality of configuration objects corresponding to the plurality of resource groups, each configuration object containing;

configuration and status information for a corresponding resource group; and

an associated owners list identifying data processing systems within the corresponding resource group;

maintaining a configuration database on each data processing system within the cluster multiprocessing system, each configuration database containing at least one configuration object,

wherein the configuration database on a data processing system contains each configuration object for a resource group which identifies the data processing system as an owner in the owners list associated with the configuration object.

9. The method of claim 8, further comprising:

replicating a configuration object on each data processing system identified within the owners list associated with the configuration object.

10. The method of claim 8, wherein the step of maintaining a configuration database on each data processing system within the cluster multiprocessing system further comprises:

maintaining, within the configuration database on a data processing system, a copy of a configuration object for each resource group including the data processing system.

11. The method of claim 10, wherein the step of maintaining, within the configuration database on a data processing system, a copy of a configuration object for each resource group including the data processing system further comprises:

maintaining a copy of two configuration objects within the configuration database on the data processing system.

12. The method of claim 8, wherein the step of instantiating a plurality of configuration objects corresponding to the plurality of resource groups further comprises:

listing, within the owners list associated with a configuration object, data processing systems permitted to alter the configuration object.

13. The method of claim 8, wherein the step of instantiating a plurality of configuration objects corresponding to the plurality of resource groups further comprises:

instantiating one configuration object for a resource group on each data processing system within the resource group.

15. The method of claim 14, wherein the step of defining a subset of data processing systems within a cluster as a resource group further comprises:

defining a highly available application and each data processing system designated to manage the application as a resource group.

16. The method of claim 15, wherein the step of defining a highly available application and each data processing system managing the application as a resource group further comprises:

defining a plurality of resource groups for each highly available application within the cluster, each resource group including all data processing systems managing the corresponding application.

17. The method of claim 14, wherein the step of defining configuration data for the resource group further comprises:

instantiating a configuration object containing configuration and status information for a highly available application corresponding to the resource group and having an associated list of data processing systems within the resource group.

18. The method of claim 17, wherein the step of replicating the configuration data only on each data processing system within the resource group further comprises:

replicating the configuration object on each data processing system identified in an owners list associated with the configuration object.

19. The method of claim 17, wherein the step of replicating the configuration data only on each data processing system within the resource group further comprises:

replicating, on a data processing system, a configuration object for each resource group including the data processing system.

20. The method of claim 14, further comprising:

maintaining, on a data processing system, a configuration object for each resource group including the data processing system and no configuration objects for other resource groups.

14. A method of partially replicating configuration information in a distributed database, comprising:

defining a subset of data processing systems within a cluster system as a resource group;

defining configuration data for the resource group; and

replicating the configuration data only on each data processing system within the resource group.

21. A data processing system, comprising:

a processor executing instructions for an application server;

a connection port permitting connection of the data processing system to a cluster system to receive request for the application server; and

a memory containing configuration information for the cluster system and configuration information for at least one resource group within the cluster system, wherein the resource group is associated with the application server, the configuration information for the at least one resource group including an identification of other data processing systems in the resource group.

1. A cluster multiprocessing system, comprising:

a plurality of data processing systems segregated into a plurality of resource groups each including an application and at least two data processing systems;

a plurality of configuration objects each corresponding to a resource group within the plurality of resource groups, each configuration object containing;

configuration and status information for the corresponding resource group; and

an associated owners list identifying data processing systems within the corresponding resource group;

a configuration database on each data processing system within the cluster multiprocessing system, each configuration database containing at least one configuration object,

wherein a configuration object for a resource group is replicated on each data processing system identified in the owners list associated with the configuration object.

2. The cluster multiprocessing system of claim 1, wherein the cluster multiprocessing system comprises N data processing systems and a resource group within the plurality of resource groups contains M data processing systems, where M is smaller than N.

3. The cluster multiprocessing system of claim 2, wherein M equals three and N is at least 1,000.

4. The cluster multiprocessing system of claim 1, wherein each configuration database on a data processing system contains a configuration object for each resource group including the data processing system.

5. The cluster multiprocessing system of claim 4, wherein a configuration database on a data processing system included within two resource groups contains two configuration objects.

6. The cluster multiprocessing system of claim 1, wherein the owners list associated with a configuration object identifies data processing systems which may alter the configuration object.

7. The cluster multiprocessing system of claim 1, wherein a configuration object may only be altered by a data processing system identified within the owners list associated with the configuration object.

22. The data processing system of claim 21, further comprising:

means, responsive to a configuration change, for determining whether the configuration change affects the cluster system or only the resource group;

means, responsive to determining that the configuration change affects the cluster system, for replicating change information to all data processing systems in the cluster system; and

means, responsive to determining that the configuration change affects only the resource group, for replicating change information only to data processing systems within the resource group.

23. A computer program product in a computer usable medium, comprising:

instructions defining a subset of data processing systems within a network as a resource group;

instructions defining configuration data for the resource group; and

instructions for replicating the configuration data only on each data processing system within the resource group.

24. The computer program product of claim 23, wherein the instructions defining a highly available application and each data processing system managing the application as a resource group further comprise:

instructions defining a plurality of resource groups for each highly available application within the network, each resource group including all data processing systems managing the corresponding application.

25. The computer program product of claim 22, wherein the instructions defining configuration data for the resource group further comprises:

instructions instantiating a configuration object containing configuration and status information for a highly available application corresponding to the resource group and having an associated list of data processing systems within the resource group.

23-1. The computer program product of claim 22, wherein the instructions defining a subset of data processing systems within a network as a resource group further comprise:

instructions defining a highly available application and each data processing system designated to manage the application as a resource group.