Highly available cluster message passing facility

• US 6,446,219 B2
• Filed: 02/06/2001
• Issued: 09/03/2002
• Est. Priority Date: 05/12/1998
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Slaughter, Gregory L., Herndon, Robert
• Primary Examiner(s): Iqbal, Nadeem
• Application Number:
  US09/778,601
• Publication Number:
  US 20010032324A1
• Time in Patent Office: 574 Days
• Field of Search: 714/4, 714/5, 714/6, 714/7, 714/8, 714/10, 714/11, 714/12, 714/13, 714/20, 714/25, 714/38, 714/41, 714/43, 709/201, 709/202, 709/203, 709/206, 709/220, 709/221, 709/242, 709/239
• US Class Current: 714/4.2
• CPC Class Codes: G06F 11/142 Reconfiguring to eliminate the error group management mechanisms in a peer-to-peer network H04L67/1044

Abstract

A cluster implements a virtual disk system that provides each node of the cluster access to each storage device of the cluster. The virtual disk system provides high availability such that a storage device may be accessed and data access requests are reliably completed even in the presence of a failure. To ensure consistent mapping and file permission data among the nodes, data are stored in a highly available cluster database. Because the cluster database provides consistent data to the nodes even in the presence of a failure, each node will have consistent mapping and file permission data. A cluster transport interface is provided that establishes links between the nodes and manages the links. Messages received by the cluster transports interface are conveyed to the destination node via one or more links. The configuration of a cluster may be modified during operation. Prior to modifying the configuration, a reconfiguration procedure suspends data access requests and waits for pending data access requests to complete. The reconfiguration is performed and the mapping is modified to reflect the new configuration. The node then updates the internal representation of the mapping and resumes issuing data access requests.

Claims

1. A first node for use in a distributed computing system, said node comprising:

a connection module configured to manage one or more inter-node links between said first node and one or more other nodes of said system; and

a configuration module coupled to said connection module, wherein said configuration module is configured to;

initialize said connection module according to a current system configuration;

establish connections between active nodes of said system via said inter-node links;

detect a change in a configuration of said system; and

reconfigure said connection module in response to detecting said change.

2. The node of claim 1, wherein said configuration module is further configured to:

establish a new connection corresponding to said change in said system, in response to detecting said change comprises the addition of a new node; and

remove a connection corresponding to said change in said system, in response to detecting said change comprises the removal of one of said other nodes.

3. The node of claim 2, wherein said configuration module is further configured to:

initiate a query in response to detecting said change in configuration; and

receive system configuration information corresponding to said change, wherein said configuration information indicates active nodes of said system and a list of inter-nodes links in said system.

4. The node of claim 1, wherein said connection module is configured to receive a data access request from a first client targeted to a second node of said other nodes, and wherein said connection module is configured to convey said request to said second node via one or more of said inter-node links in a manner transparent to said first client.

5. The node of claim 4, wherein said connection module is further configured to append an indication of a destination client of said request to said data access request prior to conveyance.

6. The node of claim 4, wherein said second node includes an instance of a first connection module corresponding to said connection module of said first node, and wherein said first connection module of said second node is configured to receive said message.

7. The node of claim 6, wherein said connection module of said first node is configured to provide a communication interface between clients on said first node and clients on said second node.

8. The node of claim 3, further comprising a cluster membership monitor configured to monitor node membership in said system.

9. The node of claim 8, further comprising a configuration database configured to store information describing inter-node links in said system.

10. The node of claim 9, wherein said configuration module is configured to access said monitor to determine said change in said system.

11. A method of configuring communication in a distributed computing system comprising a plurality of nodes, the method comprising:

determining a configuration of said distributed computing system, wherein said configuration includes active nodes of said distributed computing system and links between said active nodes;

initializing a connection module according to said system configuration, wherein said connection module is configured to manage inter-node links between nodes of said system;

establishing connections between active nodes of said system via said inter-node links;

detecting a change in said configuration of said system;

reconfiguring said connection module according to said configuration change, in response to detecting said change.

12. The method of claim 11, further comprising:

establishing a new connection corresponding to said change in said system, in response to detecting said change comprises the addition of a new node; and

removing a connection corresponding to said change in said system, in response to detecting said change comprises the removal of a node.

13. The method of claim 11, further comprising:

initiating a query in response to detecting said change in configuration; and

receiving system configuration information corresponding to said change in response to said query, wherein said configuration information indicates active nodes of said system and a list of inter-node links in said system.

14. The method of claim 13, further comprising:

receiving at said connection module a data access request from a first client targeted to a second node of said plurality of nodes; and

conveying said request to said second node via one or more of said inter-node links in a manner transparent to said first client.

15. The method of claim 14, further comprising appending an indication of a destination client of said request to said data access request prior to said conveying.

16. The method of claim 14, wherein said second node includes an instance of a first connection module corresponding to said connection module of said first node, and wherein said first connection module of said second node is configured to receive said message.

17. The method of claim 12, further comprising monitoring node membership in said system.

18. The method of claim 17, further comprising a configuration database configured to store information describing inter-node links in said system.

19. The method of claim 18, wherein said configuration module is configured to access said monitor to determine said change in said system.

20. A computer-readable storage medium comprising program instructions executable to:

determine a configuration of a distributed computing system, wherein said configuration includes active nodes of said distributed computing system and links between said active nodes;

initialize a connection module according to said system configuration, wherein said connection module is configured to manage inter-node links between nodes of said system;

establish connections between active nodes of said system via said inter-node links;

detect a change in said configuration of said system;

reconfigure said connection module according to said configuration change, in response to detecting said change.

21. The computer-readable storage medium of claim 20, wherein said program instructions are further executable to:

establish a new connection corresponding to said change in said system, responsive to detecting said change comprises the addition of a new node; and

remove a connection corresponding to said change in said system, responsive to detecting said change comprises the removal of a node.

22. The computer-readable storage medium of claim 20, wherein said program instructions are further executable to:

initiate a query in response to detecting said change in configuration; and

receive system configuration information corresponding to said change in response to said query, wherein said configuration information indicates active nodes of said system and a list of inter-node links in said system.

23. The computer-readable storage medium of claim 22, wherein said program instructions are further executable to:

receive at said connection module a data access request from a first client targeted to a second node of said plurality of nodes; and

convey said request to said second node via one or more of said inter-node links in a manner transparent to said first client.

24. The computer-readable storage medium of claim 23, wherein said program instructions are further executable to append an indication of a destination client of said request to said data access request prior to conveying said request.

25. The computer-readable storage medium of claim 23, wherein said second node includes an instance of a first connection module corresponding to said connection module of said first node, and wherein said first connection module of said second node is configured to receive said request.

26. The computer-readable storage medium of claim 21, wherein said program instructions are further executable to monitor node membership in said system.

27. The computer-readable storage medium of claim 26, further comprising a configuration database configured to store information describing inter-node links in said system.

28. The computer-readable storage medium of claim 27, wherein said program instructions are further executable to access said monitor to determine said change in said system.