Cluster communications framework using peer-to-peer connections

• US 8,601,101 B1
• Filed: 06/28/2004
• Issued: 12/03/2013
• Est. Priority Date: 06/28/2004
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Oracle America, Inc. (Oracle Corporation)
• Inventors: Singh, Servesh Pratap
• Primary Examiner(s): NGUYEN, THU HA T
• Application Number:
  US10/878,785
• Time in Patent Office: 3,445 Days
• Field of Search: 709/217, 709/249, 709223-224, 702/186, 370/254
• US Class Current: 709/223
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

Various embodiments of a system and method for each node in a cluster including a peer-to-peer communication channel compiling and maintaining its own cluster membership are disclosed. In one embodiment, each cluster node may be coupled to every other node and commonly-accessible storage through a network. When a node starts up and attempts to join the cluster, it may access cluster information data stored in the commonly-accessible storage. This information may include a list of all nodes that are potential cluster members. Based on the cluster membership information, the joining node may request a peer-to-peer connection with each potential cluster member. When such a request results in establishment of a connection, the joining node may add the name of the target node to its cluster membership list. Likewise, when a member node grants a connection to a node, it may add the joining node to its cluster membership list.

Claims

1. A system, comprising:

a cluster comprising a plurality of nodes, wherein each node comprises a server configured to execute a distributed application component, and wherein each node is coupled to a network; and

a commonly accessible storage coupled to the network and configured to store cluster membership information, wherein the commonly accessible storage is accessible by each of the plurality of nodes of the cluster;

a joining node configured to join the cluster, wherein to join the cluster, the joining node is configured on startup to;

access the commonly accessible storage to retrieve the cluster membership information; and

request a peer-to-peer connection to every one of the plurality of current nodes of the cluster based on the cluster membership information, wherein, in response to a connection failure, the joining node is further configured to retry its attempt to establish a connection with a cluster node a specified number of times.

2. The system, as recited in claim 1, wherein the joining node is configured to establish a connection with each node of the cluster according to the cluster membership information, wherein in response to a successful connection the joining node and current node each update respective cluster membership lists.

3. A system, comprising:

a cluster comprising;

a plurality of nodes, wherein each node comprises a server configured to execute a distributed application component;

wherein each node is coupled to a network;

wherein each node maintains a peer-to-peer connection with every other node of the cluster for cluster configuration and membership communications, wherein each node is configured to update both its cluster membership list and its application configuration using the same peer-to-peer connection with each other node of the cluster, respectively; and

wherein each node further comprises a cluster configuration service configured to, in response to a request to modify the cluster configuration;

perform the requested modification on the configuration of its node;

send a modification request to every other node using the peer-to-peer connections; and

if the modification is not performed successfully on all nodes, rollback the modification on all nodes.

4. The system as recited in claim 3, wherein each node is further configured to attempt to communicate with another one of the plurality of nodes periodically or aperiodically to determine that the another one of the plurality of nodes is still an active member of the cluster.

5. The system, as recited in claim 4, wherein each node is further configured to update its cluster membership list to remove the another one of the plurality of nodes in response to a failure to communicate with it.

6. The system, as recited in claim 3, wherein the cluster configuration service is further configured to, in response to a request to modify the configuration of its node:

perform the requested modification on the configuration of its node; and

send a response to a requesting node indicating success or failure for the modification using the peer-to-peer connections.

7. A method performed by a node, comprising:

accessing cluster membership information from a commonly accessible storage accessible by each of a plurality of nodes of a cluster;

requesting a respective peer-to-peer connection with every one of the cluster nodes based on the cluster membership information, wherein said requesting comprises retrying to establish the connection with a particular node a specified number of times in response to an initial request for the connection with a particular node failing; and

adding each one of the cluster nodes to a cluster membership list in response to establishing a connection with each one of the cluster nodes.

8. The method as recited in claim 7, further comprising establishing connection with a joining node and adding the joining node to the cluster membership list.

9. A method performed by a node of a cluster, comprising:

maintaining a respective peer-to-peer connection with each one of a plurality of other nodes in the cluster;

attempting to communicate with each one of the plurality of other nodes in the cluster using a corresponding peer-to-peer connection for each other node, wherein attempting to communicate with one of the nodes comprises retrying the communication a specified number of times; and

removing one of the plurality of other nodes from a cluster membership list if the attempt to communicate with it is unsuccessful.

10. A method, comprising:

a node receiving a request to modify the configuration of a cluster and in response;

modifying its configuration;

sending a modification request to all other nodes of a cluster; and

in response to detecting that the modification was not successful on one or more of the cluster nodes, rolling back the modification on all cluster nodes including itself.

11. The method as recited in claim 10, further comprising:

a node receiving a request to modify its configuration and in response;

modifying the receiving node'"'"'s configuration; and

sending a response to the requesting node indicating success or failure for the modification.

12. A computer-accessible medium comprising program instructions, wherein the program instructions are computer-executable on a node to:

access cluster membership information from a commonly accessible storage accessible by each of a plurality of nodes of a cluster;

request respective a peer-to-peer connection with every one of the cluster nodes based on the cluster membership information, wherein to establish each connection, the program instructions are further computer-executable to retry the attempt a specified number of times; and

update a cluster membership list in response to establishing each connection.

13. The computer-accessible medium, as recited in claim 12, wherein the program instructions are further computer-executable to attempt to establish a connection with each of a plurality of nodes indicated by the cluster membership information and update a cluster membership list to indicate an active cluster member if the connection is successful.

14. A computer-accessible medium comprising program instructions, wherein the program instructions are computer-executable on a node of a cluster to:

maintain a respective peer-to-peer connection with every one of a plurality of other nodes in the cluster;

attempt communications from the node to each one of the plurality of other nodes using the corresponding connection for each other node;

update a cluster membership list of the node to remove one of the plurality of other nodes in response to a failure of the communications; and

in response to a request to modify the cluster configuration;

perform the requested modification on the configuration of the node;

send a modification request to every other node using the peer-to-peer connections; and

in response to the modification not being performed successfully on all nodes, rollback the modification on all nodes.

15. The computer-accessible medium, as recited in claim 14, wherein to attempt communications from the node to each one of the plurality of other nodes, the program instructions are further computer-executable to retry each attempt a specified number of times.

16. The computer-accessible medium, as recited in claim 14, wherein the program instructions are further computer-executable to attempt communications from the node to each one of the plurality of other nodes periodically or aperiodically.

17. The computer-accessible medium, as recited in claim 14, wherein the program instructions are further computer-executable to, in response to the request to modify the configuration of the node:

receive a response from another node indicating success or failure for the modification using the corresponding peer-to-peer connection.