SVC cluster configuration node failover system and method

• US 8,495,414 B2
• Filed: 07/31/2012
• Issued: 07/23/2013
• Est. Priority Date: 09/30/2009
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: He, Hao, Sun, Ling, Zhang, Jun Wei
• Primary Examiner(s): Lottich, Joshua P
• Application Number:
  US13/563,089
• Publication Number:
  US 20120297243A1
• Time in Patent Office: 357 Days
• Field of Search: 714/4.11, 714/4.12, 714/13
• US Class Current: 714/4.11
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/2017 where memory access, memory control or I/O control functionality is redundant redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035
• CPC Class Codes: H04L 41/0668 by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

Methods, systems, and computer programs are provided for failover responses to configuration node failures in SVC clusters. An SVC cluster manages a plurality of storage devices and includes a plurality of SVCs interconnected via a network, each SVC acting as a separate node. A storage device event log is saved, with an original configuration node. An entry is made in the storage device event log each time the original configuration node receives storage device events and setting the entry to a negative value indicating a storage device event has not been handled. The entry is set to a positive value indicating the storage device event has been handled after the original configuration node notifies respective subscribed clients about those of the storage device events the original configuration node receives. A new configuration node is activated in response to configuration node failures.

Claims

1. A failover method for responding to configuration node failures in a SAN Volume Controller (SVC) cluster, wherein the SVC cluster manages a plurality of storage devices and comprises a plurality of SVCs interconnected via the internet, each SVC acting as a separate node, and at any given time, only one node within the SVC cluster functions as a configuration node to operate external services, the method comprising:

saving, with an original configuration node, a storage device event log;

making an entry in the storage device event log each time the original configuration node receives storage device events and setting the entry to a negative value indicating a storage device event has not been handled;

setting the entry to a positive value indicating the storage device event has been handled after the original configuration node notifies respective subscribed clients about those of the storage device events the original configuration node receives; and

activating a new configuration node in response to failures of the original configuration node.

2. The method according to claim 1, wherein any entries in the storage device event log contains at least the following information:

an event code;

an event description; and

an indication as to whether the event has been handled.

3. The method according to claim 1, performing the following:

obtaining client subscription information related to the storage device events managed by the SVC cluster from the plurality of storage devices with the new configuration node, wherein the plurality of storage devices are not in the original configuration node,

retrieving the storage device events from a storage device event monitoring unit with the new configuration node in response to the storage device events occurring in the plurality of storage devices managed by the SVC cluster, and

sending storage device events to subscribed clients with the new configuration node based on the obtained client subscription information.

4. The method according to claim 3, wherein the storage device event monitoring unit is in a node other than the original configuration node of the SVC cluster.

5. The method according to claim 4, wherein any of the entries in the storage device event log further includes a time of event occurrence.

6. The method according to claim 5, further comprising:

with the original configuration node, making an entry in the storage device event log each time the original configuration node receives a storage device event and recording the time at which the storage device event occurs for the time of event occurrence.

7. The method according to claim 6, further comprising:

recording the time at which a failure of the original configuration node occurs in response to the failure of the original configuration node,

recording successful activation of the new configuration node in response to a successful activation of the new configuration node,

determining which events in the storage device event log occurred between the failure time of the original configuration node and the successful activation time of the new configuration node based on the failure time of the original configuration node and the successful activation time of the new configuration node,

searching for subscribed clients for the events that occurred between the failure time of the original configuration node and the successful activation time of the new configuration node based on the obtained client subscription information stored in one of the plurality of storage devices, and

sending the corresponding events to the subscribed clients for the events that occurred between the failure time of the original configuration node and the successful activation time of the new configuration node.

8. The method according to claim 1, further comprising:

with the new configuration node, searching for entries that are set to a negative value indicating the storage device event has not been handled in the storage device event log after activation,

searching for subscribed clients who have subscribed to events corresponding to all the records based on the obtained client subscription information, and

sending the corresponding events to clients who have subscribed to these events.

9. A failover system for responding to configuration node failures in a SAN Volume Controller (SVC) cluster, the SVC cluster managing a plurality of storage devices and comprising a plurality of SVCs interconnected via the internet, each SVC acting as a separate node, and at any given time, only one node within the SVC cluster functioning as a configuration node to operate external services, the system comprising:

a storage device to save, with an original configuration node, a storage device event log;

the storage device makes an entry in the storage device event log each time the original configuration node receives storage device events and setting the entry to a negative value indicating a storage device event has not been handled;

the a storage device sets the entry to a positive value indicating the storage device event has been handled after the original configuration node notifies respective subscribed clients about those of the storage device events the original configuration node receives; and

an activation unit to activate a new configuration node in response to node failures of the original configuration node.

10. The failover system according to claim 9, wherein any entries in the storage device event log contains at least the following information:

an event code;

an event description; and

an indication as to whether the event has been handled.

11. The failover system according to claim 9, further including:

a storage device to store client subscription information regarding the storage device events managed by the SVC cluster, and the storage device is not in the original configuration node,

a storage device event monitoring unit to monitor the plurality of storage devices managed by the SVC cluster,

a first retrieval unit to enable the new configuration node to retrieve the client subscription information from the plurality of storage devices,

a second retrieval unit to enable the new configuration node to retrieve a storage device event from the storage device event monitoring unit, and

a submission device to enable the new configuration node to send the storage device event to subscribed clients based on the client subscription information.

12. The failover system according to claim 11, wherein the storage device event monitoring unit is in a node other than the original configuration node of the SVC cluster.

13. The failover system according to claim 9, wherein following the activation of the new configuration node, the new configuration node searches for all the entries that are set to a negative value indicating the storage device event has not been handled, searches for the subscribed clients who have subscribed to events corresponding to the entries that are set to a negative value indicating the storage device event has not been handled based on the client subscription information, and sends the corresponding events to the respective subscribed clients.

14. The failover system according to claim 9, wherein any of the entries in the storage device event log contains a time of event occurrence.

15. The failover system according to claim 14, wherein the storage device event monitoring unit of the original configuration node makes an entry in the event log every time the storage device event monitoring unit receives a storage device event and records the time at which the event occurred for the time of event occurrence.

16. The failover system according to claim 15, wherein:

when the activation unit monitors a failure of the original configuration node, the activation unit records the time at which the failure of the original configuration node occurs,

the new configuration node records the successful activation time of the new configuration node following successful activation of the new configuration node, and

the new configuration node determines which events in the storage device event log occurred between the failure time of the original configuration node and the successful activation time of the new configuration node based on the failure time of the original configuration node and the successful activation time of the new configuration node and searches for subscribed clients for the events that occurred between the failure time of the original configuration node and the successful activation time of the new configuration node based on the subscription information stored in the storage device and the corresponding events to the subscribed clients for the events that occurred between the failure time of the original configuration node and the successful activation time of the new configuration node.

17. A computer program product for responding to configuration node failures in a SAN Volume Controller (SVC) cluster, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion that saves, with an original configuration node, a storage device event log;

a second executable portion that makes an entry in the storage device event log each time the original configuration node receives storage device events and setting the entry to a negative value indicating a storage device event has not been handled;

a third executable portion that sets the entry to a positive value indicating the storage device event has been handled after the original configuration node notifies respective subscribed clients about those of the storage device events the original configuration node receives; and

a fourth executable portion that activates a new configuration node in response to failures of the original configuration node.

18. The computer program product according to claim 17, wherein any entries in the storage device event log contains at least the following information:

an event code;

an event description; and

an indication as to whether the event has been handled.

19. The computer program product according to claim 17, further comprising a fifth executable portion that performs the following:

obtaining client subscription information related to the storage device events managed by the SVC cluster from the plurality of storage devices with the new configuration node, wherein the plurality of storage devices are not in the original configuration node,

retrieving the storage device events from a storage device event monitoring unit with the new configuration node in response to the storage device events occurring in the plurality of storage devices managed by the SVC cluster, and

sending storage device events to subscribed clients with the new configuration node based on the obtained client subscription information.

20. The computer program product according to claim 19, wherein the storage device event monitoring unit is in a node other than the original configuration node of the SVC cluster.

21. The computer program product according to claim 20, wherein any of the entries in the storage device event log further includes a time of event occurrence.

22. The computer program product according to claim 21, further comprising a fifth executable portion that, with the original configuration node, makes an entry in the storage device event log each time the original configuration node receives a storage device event and recording the time at which the storage device event occurs for the time of event occurrence.

23. The computer program product according to claim 22, further comprising a sixth executable portion that performs the following:

recording the time at which a failure of the original configuration node occurs in response to the failure of the original configuration node,

recording successful activation of the new configuration node in response to a successful activation of the new configuration node,

determining which events in the storage device event log occurred between the failure time of the original configuration node and the successful activation time of the new configuration node based on the failure time of the original configuration node and the successful activation time of the new configuration node,

searching for subscribed clients for the events that occurred between the failure time of the original configuration node and the successful activation time of the new configuration node based on the obtained client subscription information stored in one of the plurality of storage devices, and

sending the corresponding events to the subscribed clients for the events that occurred between the failure time of the original configuration node and the successful activation time of the new configuration node.

24. The computer program product according to claim 17, further comprising a fifth executable portion that performs the following:

with the new configuration node, searching for entries that are set to a negative value indicating the storage device event has not been handled in the storage device event log after activation,

searching for subscribed clients who have subscribed to events corresponding to all the records based on the obtained client subscription information, and

sending the corresponding events to clients who have subscribed to these events.