Load balancing and fault tolerant service in a distributed data system

• US 9,785,480 B2
• Filed: 02/12/2015
• Issued: 10/10/2017
• Est. Priority Date: 02/12/2015
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Kamawat, Rakshit, TS, Veena, Marappa, Ganesh
• Primary Examiner(s): KO, CHAE M
• Application Number:
  US14/620,591
• Publication Number:
  US 20160239350A1
• Time in Patent Office: 971 Days
• Field of Search: 714 411, 714 63
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/1484 involving virtual machines
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 9/5088 involving task migration

Abstract

Techniques for load balancing and fault tolerant service are described. An apparatus may comprise load balancing and fault tolerant component operative to execute a load balancing and fault tolerant service in a distributed data system. The load balancing and fault tolerant service distributes a load of a task to a first node in a cluster of nodes using a routing table. The load balancing and fault tolerant service stores information to indicate the first node from the cluster of nodes is assigned to perform the task. The load balancing and fault tolerant service detects a failure condition for the first node. The load balancing and fault tolerant service moves the task to a second node from the cluster of nodes to perform the task for the first node upon occurrence of the failure condition.

Claims

1. A method, comprising:

distributing a load of a task to a first node in a cluster of nodes using a routing table;

replicating the routing table to the first node and a second node in the cluster of nodes using a replicated database (RDB) service;

storing information to indicate that the first node is assigned to perform the task;

detecting a failure condition for the first node; and

reassigning the task from the first node to the second node to perform the task based upon occurrence of the failure condition, wherein the first node and the second node are computers capable of executing the task.

2. The method of claim 1, comprising:

creating a relationship between the first node and the second node;

delegating the relationship from the first node to the second node for load balancing the second node based upon detection of the failure condition of the first node;

detecting a return of the first node to an active status; and

returning the relationship from the second node to the first node based upon the first node returning to the active status following the failure condition.

3. The method of claim 1, comprising:

saving routing information, associated with the routing table, in a structured format in the first node;

maintaining a quorum for the cluster of nodes; and

providing a notification upon a node joining or leaving the quorum.

4. The method of claim 1, comprising:

restarting the task by the second node upon reassignment of the task to the second node; and

restoring the load of the task to the first node based upon the first node becoming active after the failure condition.

5. The method of claim 1, comprising joining a new node, capable of receiving the load, to the cluster of nodes, wherein the new node is the second node.

6. The method of claim 1, comprising maintaining in the routing table an ownership list indicating which node in the cluster of nodes is a most recent owner responsible for the task.

7. The method of claim 1, comprising determining a performance state for the first node.

8. The method of claim 1, comprising maintaining user space processes on a plurality of nodes of the cluster of nodes for distributing the load of the task.

9. A computing device, comprising:

a memory containing computer-readable storage medium having stored thereon instructions for performing a method; and

a processor coupled with the memory, the processor configured to execute the instructions to cause the processor to;

distribute a load of a task to a first node in a cluster of nodes using a routing table;

replicate the routing table to the first node and a second node using a replicated database (RDB) service;

store information to indicate that the first node is assigned to perform the task;

detect a failure condition for the first node; and

reassign the task from the first node to the second node in the cluster of nodes to perform the task based upon occurrence of the failure condition.

10. The computing device of claim 9, the instructions to cause the processor to:

create a relationship between the first node and the second node;

delegate the relationship from the first node to the second node for load balancing the second node based upon detection of the failure condition of the first node;

detect a return of the first node to an active status; and

return the relationship from the second node to the first node based upon the first node returning to the active status following the failure condition.

11. The computing device of claim 9, the instructions to cause the processor to:

save routing information, associated with the routing table, in a structured format in the first node;

maintain a quorum of the cluster of nodes; and

provide a notification upon a node joining or leaving the quorum.

12. The computing device of claim 9, the instructions to cause the processor to:

restart the task by the second node upon occurrence of the failure condition; and

restore the load of the task to the first node based upon the first node becoming active after the failure condition.

13. The computing device of claim 9, the instructions to cause the processor to join a new node, capable of receiving the load, to the cluster of nodes, wherein the new node is the second node.

14. The computing device of claim 9, the instructions to cause the processor to:

maintain in the routing table an ownership list indicating which node in the cluster of nodes is a most recent owner responsible for the task;

determine a performance state for the first node; and

maintain user space processes on a plurality of nodes of the cluster of nodes for distributing the load of the task.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to:

distribute a load of a task to a first node in a cluster of nodes using a routing table;

replicate the routing table to the first node and a second node in the cluster of nodes using a replicated database (RDB) service;

store information to indicate that the first node is assigned to perform the task;

detect a failure condition for the first node; and

reassign the task from the first node to the second node to perform the task based upon occurrence of the failure condition, wherein the first node and the second node comprise storage controllers capable of executing the task to perform a storage operation.

16. The computer-readable storage medium of claim 15, comprising further instructions that, when executed by the processor, cause the processor to:

create a relationship between the first node and the second node;

delegate the relationship from the first node to the second node for load balancing the second node based upon detection of the failure condition of the first node;

detect a return of the first node to an active status; and

return the relationship from the second node to the first node based upon the first node returning to the active status following the failure condition.

17. The computer-readable storage medium of claim 15, comprising further instructions that, when executed by the processor, cause the processor to:

save routing information, associated with the routing table, in a structured format in the first node;

maintain a quorum for the cluster of nodes; and

provide a notification upon a node joining or leaving the quorum.

18. The computer-readable storage medium of claim 15, comprising further instructions that, when executed by the processor, cause the processor to:

restart the task by the second node upon occurrence of the failure condition; and

restore the load of the task to the first node based upon the first node becoming active after the failure condition.

19. The computer-readable storage medium of claim 15, comprising further instructions that, when executed by the processor, cause the processor to join a new node, capable of receiving the load, to the cluster of nodes, wherein the new node is the second node.

20. The computer-readable storage medium of claim 15, comprising further instructions that, when executed by the processor, cause the processor to:

maintain in the routing table an ownership list indicating which node in the cluster of nodes is a most recent owner responsible for the task;

determine a performance state for the first node; and

maintain user space processes on a plurality of nodes of the cluster of nodes for distributing the load of the task.