Method and apparatus for resolving partial connectivity in a clustered computing system

• US 6,757,836 B1
• Filed: 01/10/2000
• Issued: 06/29/2004
• Est. Priority Date: 01/10/2000
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Murphy, Declan J., Kumar, Krishna
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): MASKULINSKI, MICHAEL C
• Application Number:
  US09/479,468
• Time in Patent Office: 1,632 Days
• Field of Search: 714/4, 714/11-13, 709/102-105
• US Class Current: 714/4.1
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/2023 Failover techniques
• CPC Class Codes: G06F 11/2035 without idle spare hardware

Abstract

Improved techniques for resolving partial connectivity in clustered computing systems are disclosed. The improved techniques allow for detection of one or more disconnections that cause a loss of full connectivity and then resolution of the disconnections by shutting down one or more appropriate nodes of the clustered computing system to regain full connectivity. As a result, the clustered computing system can effectively maintain a full connectivity condition that is often needed by software running on the nodes of the clustered computing system.

Claims

1. A method for monitoring full connectivity in a clustered computing system having more than two nodes, said method comprising:

(a) detecting loss of full connectivity in the clustered computing system, wherein each of the nodes of the clustered computing system is directly connected via at least one communication link to every other node in the clustered computing system when the clustered computing system is fully connected;

(b) determining, at each of the nodes, which one or more of the nodes of the clustered computing system should be shut down to regain full connectivity in the clustered computing system; and

(c) shutting down the one or more nodes of the clustered computing system that said determining (b) has determined should be shut down to regain full connectivity in the clustered computing system, wherein each of the nodes has a unique node number, and

 

wherein said determining (b) comprises;

(b1) determining whether one of the nodes has recently been rebooted;

(b2) selecting the one of the nodes that has recently been rebooted to be shut down when said determining (b1) determines that one of the nodes has recently been rebooted;

(b3) determining a potential vote for each of the nodes;

(b4) determining whether one of the nodes has a lower potential vote than other of the nodes;

(b5) selecting the one of the nodes having the lowest potential vote to be shut down when said determining (b1) determines that one of the nodes has not recently been rebooted and said determining (b4) determines that one of the nodes has a lower potential vote than other of the nodes; and

(b6) selecting the one of the nodes based on the node number to be shut down when said determining (b1) determines that one of the nodes has not recently been rebooted and said determining (b4) determines that none of the nodes has a lower potential vote than other of the nodes.

2. A method for maintaining full connectivity in a clustered computing system having more than two nodes, said method comprising:

(a) detecting loss of full connectivity in the clustered computing system, wherein each of the nodes of the clustered computing system is directly connected via at least one communication link to every other node in the clustered computing system when the clustered computing system is in a state of full connectivity;

(b) determining which one or more of the nodes of the clustered computing system should be shut down to regain full connectivity in the clustered computing system based on at least one of reboot status of the nodes and votes associated with the nodes; and

(c) shutting down the one or more nodes of the clustered computing system that said determining (b) has determined should be shut down to regain full connectivity in the clustered computing system,

 

wherein said detecting (a) operates to identify one or more disconnections, wherein said determining (b) and said shutting down (c) operate to process each disconnection, disconnection-by-disconnection, until full connectivity is regained, wherein each of the nodes has a unique node number, and wherein said determining (b) comprises;

(b1) determining whether one of the nodes has recently been rebooted;

(b2) selecting the one of the nodes that has recently been rebooted to be shut down when said determining (b1) determines that one of the nodes has recently been rebooted;

(b3) determining a potential vote for each of the nodes;

(b4) determining whether one of the nodes has a lower potential vote than other of the nodes;

(b5) selecting the one of the nodes having the lowest potential vote to be shut down when said determining (b1) determines that one of the nodes has not recently been rebooted and said determining (b4) determines that one of the nodes has a lower potential vote than other of the nodes; and

(b6) selecting the one of the nodes based on the node number to be shut down when said determining (b1) determines that one of the nodes has not recently been rebooted and said determining (b4) determines that none of the nodes has a lower potential vote than other of the nodes.

3. A method for maintaining full connectivity in a clustered computing system having more than two nodes, said method comprising:

(a) detecting loss of full connectivity in the clustered computing system, wherein each of the nodes of the clustered computing system is directly connected via at least one communication link to every other node in the clustered computing system when the clustered computing system is in a state of full connectivity;

(b) determining which one or more of the nodes of the clustered computing system should be shut down to regain full connectivity in the clustered computing system based on at least one of reboot status of the nodes and votes associated with the nodes; and

(c) shutting down the one or more nodes of the clustered computing system that said determining (b) has determined should be shut down to regain full connectivity in the clustered computing system,

 

wherein said detecting (a) operates to identify one or more disconnections, wherein said determining (b) and said shutting down (c) operate to process each disconnection, disconnection-by-disconnection, until full connectivity is regained, wherein each of the nodes has a unique node number, wherein said determining (b) comprises;

(b1) determining whether one of the nodes has recently been rebooted;

(b2) selecting the one of the nodes that has recently been rebooted to be shut down when said determining (b1) determines that one of the nodes has recently been rebooted;

(b3) determining a potential vote for each of the nodes;

(b4) determining whether one of the nodes has a lower potential vote than other of the nodes;

(b5) selecting the one of the nodes having the lowest potential vote to be shut down when said determining (b1) determines that one of the nodes has not recently been rebooted and said determining (b4) determines that one of the nodes has a lower potential vote than other of the nodes; and

(b6) selecting the one of the nodes based on the node number to be shut down when said determining (b1) determines that one of the nodes has not recently been rebooted and said determining (b4) determines that none of the nodes has a lower potential vote than other of the nodes;

wherein said detecting (a) operates to identify one or more disconnections, and wherein said determining (b) and said shutting down (c) operate to process each disconnection, disconnection-by-disconnection, until full connectivity is regained.