Closed-loop node membership monitor for network clusters

• US 6,532,494 B1
• Filed: 05/28/1999
• Issued: 03/11/2003
• Est. Priority Date: 05/28/1999
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Kundu, Joydip, Cusson, Michael J., Frank, Richard L., OʼShaughnessy, Daniel E.
• Primary Examiner(s): Meky, Moustafa M.
• Application Number:
  US09/321,998
• Time in Patent Office: 1,383 Days
• Field of Search: 709/200, 709/201, 709/224, 709/220, 709/221, 709/239, 709/251, 709/252, 709/253
• US Class Current: 709/224
• CPC Class Codes: G06F 2209/505 Clust
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources

Abstract

A quorumless network cluster provides a highly available system by addressing the partition-in-space and partition-in-time problems in network clusters.

In a particular solution, a cluster manager (CM) can use disk based messaging to manage the operation of the cluster. Each node within the cluster must have access to a shared disk to operate within the cluster.

A particular methodology can operate the cluster in a closed loop between nodes 1 to N. If a node fails to receive a heartbeat message from its predecessor in the loop, it initiates a cluster reconfiguration by sending a reconfiguration message to each other node in the cluster.

The quorumless cluster can also include a common storage for a cluster definition. Each node may provide a proposed change to the cluster definition, however only a single coordinator node may update the cluster definition and apply the suggested changes.

Claims

1. A method for monitoring membership of a computer network cluster, the method comprising:

for each of at least three operating computing nodes, assigning a previous node and a subsequent node to form a closed messaging loop;

from each node, sending a respective heartbeat message toward the assigned subsequent node and scheduling receipt of a respective heartbeat message from the assigned previous node; and

in response to a failure of a node to receive the respective heartbeat message from the assigned previous node, reconfiguring membership of nodes in the cluster.

2. The method of claim 1 wherein sending comprises sending a single heartbeat message to the assigned subsequent node.

3. The method of claim 1 wherein reconfiguring membership comprises sending a signal from the node toward each other node belonging to the cluster.

4. The method of claim 1 wherein sending is responsive to a specific time interval.

5. The method of claim 1 wherein reconfiguring comprises:

sending a resolve heartbeat message from each node toward each other node;

at each operating node, receiving the resolve messages; and

at each operating node, configuring membership of the cluster in response to the received resolve heartbeat messages.

6. An Apparatus for monitoring membership in a network cluster, including a plurality of at least three operating computing nodes, comprising:

assignment logic for assigning each node with a previous node and a subsequent node;

for each node, membership monitor logic comprising;

a transmitter for sending an outgoing heartbeat message to the subsequent node;

a receiver for receiving an incoming heartbeat message from the previous node; and

a signal source for providing a configuration signal to each node if the receiving fails to receive the incoming heartbeat message; and

configuration logic responsive to the configuration signal for reconfiguring the membership of nodes on the cluster.

7. The apparatus of claim 6 wherein the transmitter sends a single heartbeat message to the assigned subsequent node.

8. The apparatus of claim 6 wherein the reconfiguration logic sends a signal from the node toward each other node belonging to the cluster.

9. The apparatus of claim 6 wherein the transmitter is responsive to a specific time interval.

10. The apparatus of claim 6 wherein the configuration logic comprises logic for:

sending a resolve heartbeat message from each node toward each other node;

at each operating node, receiving the resolve messages; and

at each operating node, configuring membership of the cluster in response to the received resolve heartbeat messages.

11. A computer program product for monitoring membership of a computer network cluster, the computer program product comprising a computer usable medium having computer readable code thereon, including program code which:

for each of at least three operating computing nodes, assigns a previous node and a subsequent node to form a closed messaging loop;

from each node, sends a respective heartbeat message toward the assigned subsequent node and schedules receipt of a respective heartbeat message from the assigned previous node; and

in response to a failure of a node to receive the respective heartbeat message from the assigned previous node, reconfigures membership of nodes in the cluster.

12. The computer program product of claim 11 wherein the program code further comprises instructions for sending comprises sending a single heartbeat message to the assigned subsequent node.

13. The computer program product of claim 11 wherein the instructions for reconfiguring membership comprises instructions for sending a signal from the node toward each other node belonging to the cluster.

14. The computer program product of claim 11 wherein sending is responsive to a specific time interval.

15. The computer program product of claim 11 wherein reconfiguring comprises:

sending a resolve heartbeat message from each node toward each other node;

at each operating node, receiving the resolve messages; and

at each operating node, configuring membership of the cluster in response to the received resolve heartbeat messages.

16. A method of managing node membership on a computer network cluster, comprising:

defining a logical order of at least three computing nodes in the cluster, each node having a single logically previous node and a single logically subsequent node; and

defining a heartbeat message protocol where each node in the cluster forwards a membership heartbeat message to its single logically subsequent node and schedules receipt of a membership heartbeat message from its single logically previous node.

17. The method of claim 16 further comprising reconfiguring the cluster in response to a failure of any node to receive a membership heartbeat message.

18. The method of claim 17 wherein reconfiguring comprises receiving a plurality of resolve heartbeat messages from each node, each resolve heartbeat message forwarded from a respective node.

19. The method of claim 18 wherein reconfiguring further comprises configuring membership of the cluster in response to the received resolve heartbeat messages.

20. The method of claim 16 wherein the cluster is a quorumless cluster.

21. The method of claim 16 wherein the heartbeat messaging protocol includes sending the membership heartbeat messages at a specific time.

22. The method of claim 21 wherein the specific time occurs at a specific time interval.

23. The method of claim 22 wherein the specific time interval is periodic.

24. A system for managing node membership on a computer network cluster comprising:

assignment logic for defining a logical order of at least three computing nodes in the cluster, each node having a single logically previous node and a single logically subsequent node; and

a heartbeat message protocol where each node in the cluster forwards a membership heartbeat message to its single logically subsequent node and schedules receipt of a membership heartbeat message from its single logically previous node.

25. The system of claim 24 further comprising logic for reconfiguring the cluster in response to a failure of any node to receive a membership heartbeat message.

26. The system of claim 25 wherein the reconfiguring logic comprises logic on each operating member node for resolving a plurality of resolve heartbeat messages from each node, each resolve heartbeat message forwarded from a respective node.

27. The system of claim 26 wherein the reconfiguring logic further comprises configuring membership of the cluster in response to the received resolve heartbeat messages.

28. The system of claim 24 wherein the cluster is a quorumless cluster.

29. The system of claim 24 wherein the heartbeat message protocol includes sending the membership heartbeat messages at a specific time.

30. The system of claim 29 wherein the specific time occurs at a specific time interval.

31. The system of claim 30 wherein the specific time interval is periodic.

32. A computer program product, comprising:

a computer-usable medium; and

computer program instructions carried on the medium to manage node membership on a computer network cluster, including instructions for;

defining a logical order of at least three computing nodes in the cluster, each node having a single logically previous node and a single logically subsequent node;

defining a heartbeat message protocol where each node in the cluster forwards a membership heartbeat message to its single logically subsequent node and schedules receipt of a membership heartbeat message from its single logically previous node; and

reconfiguring membership of nodes in the cluster in response to a failure of a node to receive the membership heartbeat message.

33. The computer program product of claim 32 further comprising instructions for reconfiguring the cluster in response to a failure of any node to receive a membership heartbeat message.

34. The computer program product of claim 33 wherein reconfiguring comprises receiving a plurality of resolve heartbeat messages from each node, each resolve heartbeat message forwarded from a respective node.

35. The computer program product of claim 34 wherein reconfiguring further comprises configuring membership of the cluster in response to the received resolve heartbeat messages.

36. The computer program product of claim 32 wherein the cluster is a quorumless cluster.

37. The computer program product of claim 32 wherein the heartbeat message protocol includes sending the membership heartbeat messages at a specific time.

38. The computer program product of claim 37 wherein the specific time occurs at a specific time interval.

39. The computer program product of claim 38 wherein the specific time interval is periodic.

40. A system for monitoring membership of a computer network cluster, the system comprising:

for each of at least three operating computing nodes, a means for assigning a previous node and a subsequent node to form a closed messaging loop;

from each node, a means for sending a respective heartbeat message toward the assigned subsequent node and a means for scheduling receipt of a respective heartbeat message from the assigned previous node; and

in response to a failure of a node to receive the respective heartbeat message from the assigned previous node, a means for reconfiguring membership of nodes in the cluster.

41. The system of claim 40 wherein the sending means comprises sending a single heartbeat message to the assigned subsequent node.

42. The system of claim 40 wherein the reconfiguring membership means comprises a means for sending a signal from the node toward each other node belonging to the cluster.

43. The system of claim 40 wherein the sending means is responsive to a specific time interval.

44. The system of claim 40 wherein the reconfiguring membership means comprises:

a means for sending a resolve heartbeat message from each node toward each other node;

at each operating node, a means for receiving the resolve messages; and

at each operating node, a means for configuring membership of the cluster in response to the received resolve heartbeat messages.

45. A system for managing node membership on a computer network cluster, the system comprising:

a means for defining a logical order of at least three computing nodes in the cluster, each node having a single logically previous node and a single logically subsequent node; and

a means for defining a heartbeat message protocol where each node in the cluster forwards a membership heartbeat message to its single logically subsequent node and a means for scheduling receipt of a membership heartbeat message from its single logically previous node.

46. The system of claim 45 further comprising a means for reconfiguring the cluster in response to a failure of any node to receive a membership heartbeat message.

47. The system of claim 46 wherein the reconfiguring means comprises a means for receiving a plurality of resolve heartbeat messages from each node, each resolve heartbeat message forwarded from a respective node.

48. The system of claim 47 wherein the reconfiguring means further comprises a means for configuring membership of the cluster in response to the received resolve heartbeat messages.

49. The system of claim 45 wherein the cluster is a quorumless cluster.

50. The system of claim 45 wherein the heartbeat messaging protocol includes a means for sending the membership heartbeat messages at a specific time.

51. The system of claim 50 wherein the specific time occurs at a specific time interval.

52. The system of claim 51 wherein the specific time interval is periodic.