Quorumless cluster using disk-based messaging

• US 6,871,222 B1
• Filed: 05/28/1999
• Issued: 03/22/2005
• Est. Priority Date: 05/28/1999
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Kundu, Joydip, O'Shaughnessy, Daniel E., Cusson, Michael J., Frank, Richard L.
• Primary Examiner(s): Le, N.
• Assistant Examiner(s): Benson, Walter
• Application Number:
  US09/321,090
• Time in Patent Office: 2,125 Days
• Field of Search: 707/201, 707/202, 707/203, 707/200, 707/10, 709/203, 709/218, 709/226, 714/4, 714/48, 713/201
• US Class Current: 709/223
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A quorumless network cluster provides a highly available system by addressing the partition-in-space and partition-in-time problems in network clusters. In a particular solution, a cluster manager (CM) can use disk based messaging to manage the operation of the cluster. Each node within the cluster must have access to a shared disk to operate within the cluster. A particular methodology can operate the cluster in a closed loop between nodes 1 to N. If a node fails to receive a heartbeat message from its predecessor in the loop, it initiates a cluster reconfiguration by sending a reconfiguration message to each other node in the cluster.

Claims

1. A method for managing membership of nodes in a computer network cluster, the method comprising:

defining a shareable storage device to store data for a network;

defining a message location on the shareable storage device; and

granting membership in a network cluster to a node if the node has access to the message location on the shareable storage device.

2. The method of claim 1 further comprising:

revoking membership of the node in the network cluster if the node ceases to have access to the message location on the shareable storage device.

3. The method of claim 2 further comprising:

ceasing operation of the network cluster if no node has access to the shareable storage device.

4. The method of claim 1 wherein granting membership comprises, from the node, accessing the message location in the shareable storage device.

5. The method of claim 4 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

6. The method of claim 5 wherein granting membership comprises accessing each identified physical storage device.

7. The method of claim 1 further comprising:

always granting membership in the network cluster to a node when the node has access to the shareable storage device.

8. A system for managing membership of nodes in a computer network cluster, comprising:

a network infrastructure for supporting a plurality of nodes in a network cluster;

a storage device separated from the network infrastructure and interconnectable to a plurality of nodes;

a node interconnected with the storage device; and

a manager mechanism to grant membership in the network cluster to the node based on the node having access to the storage device.

9. The system of claim 8 wherein the manager mechanism:

revokes membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

10. The system of claim 9 wherein the manager mechanism:

ceases operation of the network cluster if no node has access to the shareable storage device.

11. The system of claim 8 wherein the manager mechanism includes a message location in the shareable storage device accessed by the node being granted membership in the network cluster.

12. The system of claim 11 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

13. The system of claim 12 wherein each identified physical storage device is accessible by the node being granted membership in the network cluster.

14. The system of claim 8 wherein the manager mechanism always grants membership in the network cluster to the node exclusively based on the node having access to the storage device.

15. A computer program product for managing membership of nodes in a computer network cluster, the computer program product comprising a computer usable medium having computer readable code thereon, including program code that:

defines a shareable storage device to store data for a network cluster;

defines a message location on the shareable storage device; and

grants membership in the network cluster to a node if the node has access to the message location on the shareable storage device.

16. The computer program product of claim 15 further comprising program code that:

revokes membership of the node in the network cluster if the node ceases to have access to the message location on the shareable storage device.

17. The computer program product of claim 16 further comprises program code that:

ceases operation of the network cluster if no node has access to the shareable storage device.

18. The computer program product of claim 15 wherein granting membership comprises, from the node, accessing the message location in the shareable storage device.

19. The computer program product of claim 18 wherein the message location identifies the cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

20. The computer program product of claim 19 wherein granting membership comprises accessing each identified physical storage device.

21. The computer program of claim 15 further includes program code to:

always grant membership in the network cluster to a node if the node has access to the shareable storage device.

22. A method for managing membership of nodes in a computer network cluster, the method comprising:

defining a shareable storage device to store data for a network;

creating a message location on the shareable storage device; and

granting membership in a network cluster to a node if the node has access to the shareable storage device, using the message location.

23. The method of claim 22 further comprising:

revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

24. The method of claim 23 further comprising:

ceasing operation of the network cluster if no node has access to the shareable storage device.

25. The method of claim 22 wherein granting membership comprises, from the node, accessing the message location in the shareable storage device.

26. The method of claim 25 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

27. The method of claim 26 wherein granting membership comprises accessing each identified physical storage device.

28. A system for managing membership of nodes in a computer network cluster, comprising:

a network infrastructure for supporting a plurality of nodes in a network cluster;

a storage device separated from the network infrastructure and interconnectable to a plurality of nodes;

a message location on the shareable storage device;

a node interconnected with the storage device; and

a manager mechanism to grant membership in the network cluster to the node based on the node having access to the storage device, using the message location.

29. The system of claim 28 wherein the manager mechanism:

revokes membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

30. The system of claim 29 wherein the manager mechanism:

ceases operation of the network cluster if no node has access to the shareable storage device.

31. The system of claim 28 wherein the manager mechanism includes a message location in the shareable storage device accessed by the node being granted membership in the network cluster.

32. The system of claim 31 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

33. The system of claim 32 wherein each identified physical storage device is accessible by the node being granted membership in the network cluster.

34. A computer program product for managing membership of nodes in a computer network cluster, the computer program product comprising a computer usable medium having computer readable code thereon, including program code that:

defines a shareable storage device to store data for a network cluster;

creates a message location on the shareable storage device; and

uses the message location to grant membership in the network cluster to a node if the node has access to the shareable storage device.

35. The computer program product of claim 34 further includes computer readable code for revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

36. The computer program product of claim 35 further includes computer readable code for ceasing operation of the network cluster if no node has access to the shareable storage device.

37. The computer program product of claim 36 further including computer readable code which grants a node membership, in response to the node obtaining access to the shareable storage device.

38. The computer program product of claim 37 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

39. The computer program product of claim 38 wherein granting membership further includes accessing each identified physical storage device.

40. A system for managing membership of nodes in a computer network cluster, the system comprising:

a means for defining a shareable storage device to store data for a network;

a means for creating a message location on the shareable storage device; and

a means for granting membership in a network cluster to a node if the node has access to the shareable storage device, using the message location.

41. A method for managing membership of nodes in a computer network cluster, the method comprising:

defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster; and

monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including adding a node to the computer network cluster in response to the node obtaining access to the shareable storage device.

42. The method of claim 41 further comprising:

revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

43. The method of claim 42 further comprising:

ceasing operation of the network cluster if no node has access to the shareable storage device.

44. The method of claim 41 wherein granting membership comprises, from the node, accessing a message location in the shareable storage device.

45. The method of claim 44 wherein the message location identifies the cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

46. The method of claim 45 wherein granting membership comprises accessing each identified physical storage device.

47. A system for managing membership of nodes in a computer network cluster, the system comprising:

a network infrastructure for supporting a plurality of nodes in a computer network cluster;

a shareable storage device separated from the network infrastructure and interconnectable to the plurality of nodes;

a message location on the shareable storage device;

a node interconnected with the shareable storage device; and

a manager mechanism for monitoring node membership in the computer network cluster based on the accessibility of the message location on the shareable storage device to each node, including removing a node from the computer network cluster in response to the node losing access to the shareable storage device.

48. The system of claim 47 wherein the manager mechanism:

revokes membership of the node in the network cluster if the node ceases to have access to the message location on the shareable storage device.

49. The system of claim 48 wherein the manager mechanism:

ceases operation of the network cluster if no node has access to the shareable storage device.

50. The system of claim 47 wherein the manager mechanism includes a message location in the shareable storage device accessed by the node being granted membership in the network cluster.

51. The system of claim 50 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

52. The system of claim 51 wherein each identified physical storage device is accessible by the node being granted membership in the network cluster.

53. A system for managing membership of nodes in a computer network cluster, the system comprising:

a means for defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster; and

a means for monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including adding a node to the computer network cluster in response to the node obtaining access to the shareable storage device.

54. A system for managing membership of nodes in a computer network cluster, the system comprising:

a means for defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster; and

a means for monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including removing a node from the computer network cluster in response to the node losing access to the shareable storage device.

55. A computer program product comprising:

a computer usable medium having computer readable code thereon, including program code for managing membership of nodes in a computer network cluster by;

defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster; and

monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including removing a node from the computer network cluster in response to the node losing access to the shareable storage device.

56. The computer program product of claim 55 wherein the computer readable code that removes a node from the computer network cluster further includes computer readable code revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

57. The computer program product of claim 56 further includes computer readable code for ceasing operation of the network cluster if no node has access to the shareable storage device.

58. The computer program product of claim 55 further including computer readable code which grants a node membership, in response to the node obtaining access to a message location on the shareable storage device.

59. The computer program product of claim 58 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

60. The computer program product of claim 59 wherein granting membership further includes accessing each identified physical storage device.

61. A system for managing membership of nodes in a computer network cluster, the system comprising:

a means for defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster;

a means for defining a message location on the shareable storage device; and

a means for monitoring node membership in the computer network cluster based on the accessibility of the message location on the shareable storage device to each node, including removing a node from the computer network cluster in response to the node losing access to the shareable storage device.

62. A method for managing membership of nodes in a computer network cluster, the method comprising:

defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each member node in a computer network cluster; and

regardless of network connectivity, denying membership in the computer network cluster to a node if the node is unable to access the shareable storage device.

63. The method of claim 62 further comprising:

granting membership to the node if the node has access to the shareable storage device.

64. The method of claim 63 further comprising:

revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

65. The method of claim 63 wherein granting membership comprises, from the node, accessing a message location in the shareable storage device.

66. The method of claim 65 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

67. The method of claim 66 wherein granting membership comprises accessing each identified physical storage device.

68. The method of claim 62 further comprising:

ceasing operation of the network cluster if no node has access to the shareable storage device.

69. A computer program product for managing membership of nodes in a computer network cluster, the computer program product comprising a computer usable medium including program code for:

defining a shareable storage device to store data for a network cluster; and

denying membership in the network cluster to a node if the node has no access to the shareable storage device regardless of network connectivity of the node.

70. The computer program product of claim 69 further comprising program code for:

granting membership to the node if the node has access to the shareable storage device.

71. The computer program product of claim 69 further comprises program code for:

ceasing operation of the network cluster if no node has access to the shareable storage device.

72. A system for managing membership of nodes in a computer network cluster, comprising:

a network infrastructure for supporting a plurality of nodes in a network cluster;

a storage device separated from the network infrastructure and interconnectable to a plurality of nodes;

a node interconnected with the storage device; and

a manager mechanism to grant membership in the network cluster to the node based on the node having access to the storage device regardless of network connectivity of the node.

73. The system of claim 72 wherein the manager mechanism:

grants membership to the node if the node has access to the shareable storage device.

74. The system of claim 73 wherein the manager mechanism:

revokes membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

75. The system of claim 74 wherein the manager mechanism includes a message location in the shareable storage device accessed by the node being granted membership in the network cluster.

76. The system of claim 75 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

77. The system of claim 76 wherein each identified physical storage device is accessible by the node being granted membership in the network cluster.

78. The system of claim 72 wherein the manager mechanism:

ceases operation of the network cluster if no node has access to the shareable storage device.

79. A system for managing membership of nodes in a computer network cluster, the system comprising:

a means for defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each member node in a computer network cluster; and

a means for denying membership in the computer network cluster to a node if the node has no access to the shareable storage device regardless of network connectivity of the node.

80. A method for managing membership of nodes in a computer network cluster, the system comprising:

defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster; and

monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including removing a node from the computer network cluster in response to the node losing access to the shareable storage device.

81. The method of claim 80 wherein removing a node from the computer network cluster in response to the node losing access to the shareable storage device further includes:

revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

82. The method of claim 81 further including:

ceasing operation of the network cluster if no node has access to the shareable storage device.

83. The method of claim 80 further including:

granting membership comprises to a node if the node has access to the shareable storage device using a message location in the shareable storage device.

84. The method of claim 83 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

85. The method of claim 84 wherein granting membership comprises accessing each identified physical storage device.

86. A system for managing membership of nodes in a computer network cluster, the system comprising:

a network infrastructure for supporting a plurality of nodes in a computer network cluster;

a shareable storage device separated from the network infrastructure and interconnectable to the plurality of nodes;

a node interconnected with the shareable storage device; and

a manager mechanism for monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including removing a node from the computer network cluster in response to the node losing access to the shareable storage device.

87. The system of claim 86 wherein the manager mechanism:

revokes membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

88. The system of claim 87 wherein the manager mechanism:

ceases operation of the network cluster if no node has access to the shareable storage device.

89. The system of claim 86 wherein the manager mechanism includes a message location in the shareable storage device accessed by a node being granted membership in the network cluster.

90. The system of claim 89 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

91. The system of claim 90 wherein each identified physical storage device is accessible by the node being granted membership in the network cluster.

92. A computer program product including comprising:

computer readable code including instructions for managing membership of nodes in a computer network cluster by;

defining a shareable storage device to store data for a computer network, the shareable storage device being accessible by each node in a computer network cluster; and

monitoring node membership in the computer network cluster based on the accessibility of the shareable storage device to each node, including adding a node to the computer network cluster in response to the node obtaining access to the shareable storage device.

93. The computer program product of claim 92 further includes computer readable code for revoking membership of the node in the network cluster if the node ceases to have access to the shareable storage device.

94. The computer program product of claim 93 further includes computer readable code for ceasing operation of the network cluster if no node has access to the shareable storage device.

95. The computer program product of claim 92 wherein the computer readable code which adds a node to the computer network cluster in response to the node obtaining access to the shareable storage device further includes computer readable code to grant membership to the node.

96. The computer program product of claim 95 wherein the message location identifies the network cluster by identifying at least one physical storage device from the shareable storage device that is shared by the cluster member nodes.

97. The computer program product of claim 96 wherein granting membership further includes accessing each identified physical storage device.