Method and system for forming a cluster of networked nodes

• US 8,055,735 B2
• Filed: 10/15/2008
• Issued: 11/08/2011
• Est. Priority Date: 10/30/2007
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Krishnappa, Nagendra, Chakravarti, Avishek, Yarangatta, Asha Ramamurthy
• Primary Examiner(s): Hamza; Faruk
• Assistant Examiner(s): ZHANG, SHIRLEY X
• Application Number:
  US12/252,190
• Publication Number:
  US 20090113034A1
• Time in Patent Office: 1,119 Days
• Field of Search: 709220-226
• US Class Current: 709/220
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B

Abstract

A system and a method for automatic cluster formation by automatically selecting nodes based on a selection criteria configured by the user is disclosed. During a cluster re-formation process, a failed node may automatically get replaced by one of the healthy nodes in its network, which is not part of any cluster avoiding cluster failures. In case of a cluster getting reduced to a single node due to node failures and failed nodes could not be replaced due to non-availability of free nodes, the single node cluster may be merged with a existing healthy clusters in the network providing a constant level of high availability. The proposed method may also provide an affective load balancing by maintaining a constant number of member nodes in a cluster by automatically replacing the dead nodes with a healthy node.

Claims

1. A method for forming a dynamic cluster with a plurality of nodes in a network, comprising:

identifying a free node in the plurality of nodes by a remote inter cluster manager residing in a remote cluster monitor node;

copying a configuration file to the free node by the remote inter cluster manager, wherein the configuration file comprises information selected from the group consisting of node selection criteria and a required number of nodes of the plurality of nodes;

acquiring, by a local cluster manager residing in the free node, the required number of nodes for forming the dynamic cluster based on the node selection criteria in the configuration file; and

forming the dynamic cluster using the acquired nodes and the free node by the local cluster manager, wherein forming the dynamic cluster using the acquired or nodes and the free node comprises;

electing a cluster monitor node from a group consisting of the acquired nodes and the free node; and

sending, by the cluster monitor node, a message to each of remaining nodes in the group to join the dynamic cluster; and

registering the dynamic cluster with a quorum server and acquiring a lock disk from the quorum server to prevent a formation of a duplicate cluster in the network.

2. The method of claim 1, wherein the cluster monitor runs cluster services and wherein the cluster services comprise an inter cluster manager, the local cluster manager, quorum services, a local package manager and a local resource manager, wherein the quorum services comprise information to prevent the formation of the duplicate cluster in the network.

3. The method of claim 1 further comprising starting the package services on the dynamic cluster, wherein the package services include starting, stopping or monitoring of an application.

4. The method of claim 3 further comprising registering cluster information and package information running on the dynamic cluster with the quorum server.

5. The method of claim 1, wherein the request to create a dynamic cluster in the network is initiated by a cluster administrator.

6. The method of claim 1, wherein the required number of nodes are acquired based on a minimum hardware configuration.

7. The method of claim 1, wherein the required number of nodes are acquired based on a prioritized list of nodes provided by an administrator.

8. The method of claim 1, wherein the required number of nodes are acquired based on suggestions from a work load manager.

9. The method of claim 1, wherein the required number of nodes are acquired randomly from available nodes in the network.

10. The method of claim 1, wherein acquiring the required number of nodes comprises:

determining the node selection criteria from the configuration file;

sending a request to join to available nodes in the network;

parsing incoming replies in response to the request based on the node selection criteria; and

building a list of candidate nodes.

11. The method of claim 10 further comprising if there are no nodes available in the network then forcibly acquiring nodes from a low priority cluster.

12. The method of claim 10, wherein the node receiving a request to start cluster is a single node cluster in a panic state, wherein a cluster is said to be in the panic state, when the cluster reduces to the single node cluster.

13. The method of claim 10, wherein the node selection criteria comprise one or more of minimum hardware configuration, prioritized list of potential nodes provided by an administrator, nodes suggested by work load managers and nodes acquired randomly from available nodes in the network.

17. The method of claim 1, further comprising merging a node in a panic state with the dynamic cluster, comprising:

broadcasting, by the node in the panic state, a request to merge including a package weight to each cluster in the network, wherein a cluster is said to be in the panic state, when the cluster reduces to a single node;

receiving, by the dynamic cluster, the request to merge from the node in the panic state in the network;

checking, by the dynamic cluster, the ability to handle the package weight of the node in the panic state; and

sending, by the dynamic cluster, a ready to merge message to the node in the panic state and reforming the dynamic cluster with the node in the panic state as new cluster member.

18. The method of claim 17, wherein the node in the panic state is a cluster with reduced number of nodes.

24. The method of claim 1, wherein the quorum server includes a database containing details of all clusters in the network.

14. A method for forming a cluster after booting or re-booting a node, comprising:

broadcasting, by the node, a message requesting a status of each cluster in a network;

receiving and parsing, by the node, replies from cluster monitor nodes of each cluster in response to the requested message;

determining a list of clusters that are down based on the replies received from the cluster monitor nodes, wherein the node requests a configuration file from the cluster;

selecting a cluster with a high priority from the list of clusters;

acquiring a required number of nodes for the selected cluster based on parameters in the configuration file;

reforming the selected cluster by starting cluster services on the acquired nodes;

electing a cluster monitor node from the reformed cluster;

broadcasting, by the cluster monitor node, a new status of the reformed cluster to other clusters in the network for updating respective configuration files associated with each of the other clusters; and

registering the reformed cluster with a quorum server and acquiring a lock disk from the quorum server to prevent a formation of a duplicate cluster in the network.

15. The method of claim 14, wherein parsing the replies comprises comparing the replies from the cluster monitor nodes with a configuration file stored on the node.

16. The method of claim 14, wherein the cluster is selected based on a priority configured with the clusters.

19. A system for forming a dynamic cluster with at least one another node comprising:

a network;

a plurality of nodes in the network, wherein each node comprises one of a server computer and a computing device; and

a remote cluster monitor node including a remote inter cluster manager for identifying a free node in the plurality of nodes and copying a cluster configuration file to the free node, wherein the cluster configuration file comprises information selected from the group consisting of node selection criteria and a required number of nodes of the plurality of nodes, wherein the free node comprises;

a local cluster manager stored therein for acquiring the required number of nodes for forming the dynamic cluster based on the node selection criteria in the cluster configuration file and forming the dynamic cluster by electing a cluster monitor node from a group consisting of the acquired nodes and the free node, and

wherein the cluster monitor node sends a message to each of remaining nodes in the group to join the dynamic cluster, and

wherein the cluster monitor node registers the dynamic cluster with a quorum server and acquires a lock disk from the quorum server to prevent a formation of a duplicate cluster in the network.

20. The system of claim 19 wherein the local cluster manager comprises:

local package manager for managing starting, stopping and reacting to changes in packages; and

local resource manager for monitoring health of the network interface, IP address services, reporting event management service resource value changes and activation support for logical volume manager (LVM) groups.

21. A computer program product for forming a dynamic cluster with a plurality of nodes in a network, the computer program product comprising a non-transitory computer readable storage medium readable by a processing circuit and storing instruction for execution by a processing circuit for performing a method comprising:

identifying a few node in the plurality of nodes by a remote inter cluster manager residing in a remote cluster monitor node;

copying a configuration file to the free node by the remote inter cluster manager, wherein the configuration file comprises information selected from the group consisting of node selection criteria and a required number of nodes of the plurality of nodes;

acquiring, by a local cluster manager residing in the free node, the node, the required number of nodes for forming the dynamic cluster based on the node selection criteria in the configuration file; and

forming the dynamic cluster using the acquired nodes and the free node by the local cluster manager, wherein forming the dynamic cluster using the acquired or nodes and the free node comprises;

electing a cluster monitor node from a group consisting of the acquired nodes and the free node; and

sending, by the cluster monitor node, a message to each of remaining nodes in the group to join the dynamic cluster; and

registering the dynamic cluster with a quorum server and acquiring a lock disk from the quorum server to prevent a formation of a duplicate cluster in the network.

22. The computer program product of claim 21 further comprising starting the package services on the dynamic cluster, wherein the package services include starting, stopping or monitoring of an application.

23. The computer program product of claim 21, wherein acquiring the required number of nodes comprises:

determining the node selection criteria from the configuration file;

sending a request to join to available nodes in the network;

parsing incoming replies in response to the request based on the node selection criteria; and

building a list of candidate nodes.