Cluster-wide service agents

• US 9,514,010 B2
• Filed: 09/19/2014
• Issued: 12/06/2016
• Est. Priority Date: 09/19/2014
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Buzzard, Gregory, Cady, Justin Travis, Snyder, Thomas Gilbert, Venneti, Satya R., Yucel, Sakir
• Primary Examiner(s): KO, CHAE M
• Application Number:
  US14/491,912
• Publication Number:
  US 20160085645A1
• Time in Patent Office: 809 Days
• Field of Search: 714/6.3
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2017 where memory access, memory control or I/O control functionality is redundant redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2092 Techniques of failing over between control units
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

One or more techniques and/or systems are provided for cluster configuration information replication, managing cluster-wide service agents, and/or for cluster-wide outage detection. In an example of cluster configuration information replication, a replication workflow corresponding to a storage operation implemented for a storage object (e.g., renaming of a volume) of a first cluster may be transferred to a second storage cluster for selectively implementation. In an example of managing cluster-wide service agents, cluster-wide service agents are deployed to nodes of a cluster storage environment, where a master agent actively processes cluster service calls and standby agents passively wait for reassignment as a failover master in the event the master agent fails. In an example of cluster-wide outage detection, a cluster-wide outage may be determined for a cluster storage environment based upon a number of inaccessible nodes satisfying a cluster outage detection metric.

Claims

1. A system for managing cluster-wide service agents, comprising:

an agent management component configured to;

deploy a first cluster-wide service agent to a first node within a cluster storage environment;

deploy a second cluster-wide service agent to a second node within the cluster storage environment;

assign the first cluster-wide service agent as a master agent configured to actively process cluster service calls;

maintain a storage cluster service data cache for use by the master agent;

assign a first IP address of the first cluster-wide service agent, as the master agent, to the storage cluster service data cache;

assign the second cluster-wide service agent as a standby agent configured to passively wait for reassignment as a failover master agent; and

in response to identification of a failure of the master agent;

reassign the first cluster-wide service agent as the standby agent; and

reassign the second cluster-wide service agent as the failover master agent configured to actively process cluster service calls.

2. The system of claim 1, the agent management component configured to:

automatically reassign the second cluster-wide service agent as the failover master agent in real-time based upon a service interruption tolerance metric to mitigate service interruption of cluster service calls.

3. The system of claim 1, the first cluster-wide service agent comprising a first physical process configured to actively process API calls for a set of storage cluster services while having a master agent assignment.

4. The system of claim 3, the second cluster-wide service agent comprising a second physical process, as a second instance of the first physical process, configured to actively process API calls for the set of storage cluster services while having the master agent assignment.

5. The system of claim 1, the agent management component configured to:

instruct the second cluster-wide service agent, as the failover master agent, to perform a clean-up of prior cluster service call processing that was in progress by the first cluster-wide service agent before the failure.

6. The system of claim 1, the agent management component configured to:

expose an interface, comprising at least one of an initialization interface, a shutdown interface, an acquire master duties interface, or a release master duties interface, to the first cluster-wide service agent and the second cluster-wide service agent for a set of storage cluster services.

7. The system of claim 6, the interface exposed according to an asynchronous and idempotent implementation.

8. The system of claim 1, the agent management component configured to:

responsive to identifying the failure of the master agent;

purge the storage cluster service data cache to remove the first IP address from the storage cluster service data cache;

assign a second IP address of the second cluster-wide service agent, as the failover master agent, to the storage cluster service data cache; and

maintain the storage cluster service data cache for use by the failover master agent.

9. The system of claim 8, the agent management component configured to:

retarget an inter process remote procedure call to the failover master agent based upon the assignment of the second IP address to the storage cluster service data cache.

10. The system of claim 9, the agent management component configured to:

retarget an inter process remote procedure call according to an original network configuration of the cluster storage environment before the failure of the master agent.

11. The system of claim 1, the agent management component configured to:

define a sync point for the master agent.

12. The system of claim 1, the agent management component configured to:

deploy cluster-wide service agents to respective nodes within the cluster storage environment; and

enforce a single master policy for the cluster-wide service agents.

13. The system of claim 1, the agent management component configured to:

evaluate a first set of operational statistics for the first cluster-wide service agent against a second set of operational statistics for the second cluster-wide service agent to award the first cluster-wide service agent as the master agent.

14. The system of claim 1, the agent management component configured to:

identify the failure based upon a first set of operational statistics for the first cluster-wide service agent indicating that the first cluster-wide service agent is not satisfying a performance criteria.

15. A method for managing cluster-wide service agents, comprising:

deploying a first cluster-wide service agent to a first node within a cluster storage environment;

deploying a second cluster-wide service agent to a second node within the cluster storage environment;

assigning the first cluster-wide service agent as a master agent configured to actively process cluster service calls;

maintaining a storage cluster service data cache for use by the master agent;

assigning a first IP address of the first cluster-wide service agent, as the master agent, to the storage cluster service data cache;

assigning the second cluster-wide service agent as a standby agent configured to passively wait for reassignment as a failover master agent; and

responsive to identifying a failure of the master agent;

reassigning the first cluster-wide service agent as the standby agent; and

reassigning the second cluster-wide service agent as the failover master agent configured to actively process cluster service calls.

16. The method of claim 15, the reassigning the second cluster-wide service agent as the failover master agent comprising:

automatically reassigning the second cluster-wide service agent as the failover master agent in real-time based upon a service interruption tolerance metric to mitigate service interruption of cluster service calls.

17. The method of claim 15, comprising:

evaluating a first set of operational statistics for the first cluster-wide service agent against a second set of operational statistics for the second cluster-wide service agent to award the first cluster-wide service agent as the master agent.

18. The method of claim 15, comprising:

responsive to identifying the failure of the master agent;

purging the storage cluster service data cache to remove the first IP address from the storage cluster service data cache;

assigning a second IP address of the second cluster-wide service agent, as the failover master agent, to the storage cluster service data cache; and

maintaining the storage cluster service data cache for use by the failover master agent.

19. At least one non-transitory computer-readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to:

deploy a first cluster-wide service agent to a first node within a cluster storage environment;

deploy a second cluster-wide service agent to a second node within the cluster storage environment;

assign the first cluster-wide service agent as a master agent configured to actively process cluster service calls;

maintain a storage cluster service data cache for use by the master agent;

assign a first IP address of the first cluster-wide service agent, as the master agent, to the storage cluster service data cache;

assign the second cluster-wide service agent as a standby agent configured to passively wait for reassignment as a failover master agent; and

in response to identification of a failure of the master agent;

reassign the first cluster-wide service agent as the standby agent; and

reassign the second cluster-wide service agent as the failover master agent configured to actively process cluster service calls.