Method providing server affinity and client stickiness in a server load balancing device without TCP termination and without keeping flow states

• US 7,490,164 B2
• Filed: 01/23/2006
• Issued: 02/10/2009
• Est. Priority Date: 03/19/2002
• Status: Expired due to Term

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Srivastava, Sunil K.
• Primary Examiner(s): Follansbee; John
• Assistant Examiner(s): MADAMBA, GLENFORD J
• Application Number:
  US11/338,340
• Publication Number:
  US 20060130064A1
• Time in Patent Office: 1,114 Days
• Field of Search: 709/236, 709/238, 709/220, 709/224, 709/226, 709/228
• US Class Current: 709/238
• CPC Class Codes: H04L 45/50 using label swapping, e.g. multi-protocol label switch [MPLS]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/1021 based on client or server locations

Abstract

A method of routing data from a client through one or more load-balancing routers to a selected load-balanced server among a plurality of servers in a network involves: receiving, at a load balancing node in a path from the client to the plurality of servers, a first packet of a request from a client; creating and storing a mapping of flow identifying information, associated with the first packet, to a client stickiness identifier; pushing the client stickiness label into a sending path list that is stored in association with the first packet; storing the client stickiness label in a mapping of client stickiness labels to server identifiers at a last load balancing node associated with the plurality of servers, wherein the mapping associates the client stickiness label with a server identifier that uniquely identifies a selected server that has been selected from among the plurality of servers to receive the client request; and forwarding all subsequent packets associated with the client request to the same selected server based on looking up the client stickiness label of such subsequent packets in the mapping. As a result, packet flows are rapidly routed from the same client to the same server without time-consuming hop-by-hop routing decisions or repeated load-balancing decisions.

Claims

1. A method of routing data from a client through one or more load-balancing routers to a selected load-balanced server among a plurality of servers in a network, the method comprising the steps of:

receiving, at a load balancing node in a path from the client to the plurality of servers, a first packet of a request from a client;

creating and storing a mapping of flow identifying information, associated with the first packet, to a client stickiness label and a sending path list, wherein the sending path list identifies a path in the network from the client to a selected server;

storing identifiers of egress interfaces on which the first packet arrives, in the sending path list, as the first packet of the client request is routed to the plurality of servers using hop-by-hop routing decisions; and

forwarding all subsequent packets associated with the client request to the selected load-balanced server based on looking up the client stickiness label of such subsequent packets in the mapping and without hop-by-hop routing decisions.

2. A method as recited in claim 1, further comprising the steps of:

pushing the client stickiness label into the sending path list at a last load-balancing node associated with the plurality of servers, in response to receiving the first packet of the client request at the last node,

receiving one or more packets of a server response to the client request;

popping the client stickiness label from the sending path list at the last node in response to receiving packets of the server response at the last node, and

forwarding the packets of the server response to the client based on the client stickiness label.

3. A method as recited in claim 1, further comprising the steps of:

storing one or more identifiers of ingress interfaces on which the packet arrives, in a return path list, as the first packet of the client request is routed to the plurality of servers using hop-by-hop decisions;

pushing a client stickiness label into the return path list at a last load-balancing node associated with the plurality of servers, in response to receiving the first packet of the client request at the last node,

receiving one or more packets of a server response to the client request;

popping the client stickiness label from the return path at the last node in response to receiving packets of the server response at the last node, and

forwarding the packets of the server response to the client based on the client stickiness label.

4. A method as recited in claim 1, further comprising the step of setting a first flag value in the first packet of the client request.

5. A method as recited in claim 1, further comprising the step of setting a first flag value in the first packet of the client request, wherein the first flag value signals that all nodes should update the sending path list.

6. A method as recited in claim 1, further comprising the step of setting a second flag value in each of the subsequent packets.

7. A method as recited in claim 5, wherein the second flag value instructs nodes to use the sending path list for routing the subsequent packets for server load balancing.

8. A method as recited in claim 4, wherein the first flag value signals that the packet contains the sending path list and comprises an IP-RECORD-ROUTE-OPTION-FOR-SLB flag value in a header of the first packet.

9. A method as recited in claim 5, wherein the second flag value instructs nodes to use the sending path list for routing the subsequent packets and comprises an IP-STRICT-ROUTE-OPTION-FOR-SLB flag value in a header of the subsequent packets.

10. A method as recited in claim 1, further comprising the step of selecting one of the servers to respond to the client request in response to receiving the first packet of the client request at a first load-balancing node in the network associated with the client.

11. A method as recited in claim 1, wherein the client stickiness label uniquely identifies an association of the client to the selected server.

12. A method as recited in claim 1, further comprising storing the sending path list in a mapping at a last load balancing node when the first packet reaches the last node, wherein the mapping comprises an association of sending path lists and client identifying information.

13. A method as recited in claim 1, further comprising storing the sending path list in a mapping at a last load balancing node when the first packet reaches the last node, wherein the mapping comprises an association of sending path lists and packet flow identifying information.

14. A method as recited in claim 1, further comprising the steps of storing one or more identifiers of ingress interfaces on which the first packet of the client request arrives, in a return path list, as the first packet of the client request is routed from a first load balancing node to a last load balancing node using hop-by-hop decisions, based on setting an IP-RECORD-ROUTE-OPTION-FOR-SLB flag value in the first packet of the client request.

15. A method as recited in claim 14, further comprising the steps of:

receiving one or more subsequent packets of the server reply;

setting an IP-STRICT-ROUTE-OPTION-FOR-SLB flag value in each of the subsequent packets of the server reply; and

forwarding the subsequent packets of the server reply to the client on a route that is defined only by the return path list and without hop-by-hop routing decisions.

16. A method as recited in claim 14, further comprising storing the return path list in a mapping at the first node when the packet reaches the first node, wherein the mapping comprises an association of return path lists and server identifiers.

17. A method as recited in claim 16, further comprising the steps of popping a server identifier from the sending path list and forwarding the packet to the server on an IP address contained in the server identifier.

18. A method as recited in claim 16, further comprising the steps of popping a server identifier from the sending path list, performing network address translation on the server identifier to determine where to forward the packet, and forwarding the packet to the server on an IP address determined by the network address translation.

19. A method as recited in claim 14, wherein the first node and last node are load-balancing routers that participate in the network as multicast nodes.

20. A computer-readable storage medium storing one or more sequences of instructions for routing data from a client through one or more load-balancing nodes to a selected load-balanced server among a plurality of servers in a network, which instructions, when executed by one or more processors, cause the one or more processors to carry out the steps of:

receiving, at a load balancing node in a path from the client to the plurality of servers, a first packet of a request from a client;

creating and storing a mapping of flow identifying information, associated with the first packet, to a client stickiness label and a send path list, wherein the send path list identifies a path in the network from the client to a selected server; and

forwarding all subsequent packets associated with the client request to the selected load-balanced server based on looking up the client stickiness label of such subsequent packets in the mapping.

21. An apparatus for routing data from a client through one or more load-balancing nodes to a selected load-balanced server among a plurality of servers in a network, comprising:

means for receiving, at a last load balancing node associated with a selected server among the plurality of servers, a first packet of a server reply to a request from a client;

means for storing one or more identifiers of ingress interfaces on which the packet arrives, in a send path list;

means for setting a first flag value in the first packet of the server reply, wherein the first flag indicates that all nodes, of the load-balancing nodes, should update the send path list as the first packet is routed from the last load balancing node to the client using hop-by-hop routing decisions;

means for receiving one or more subsequent packets of the client request;

means for setting a second flag value in each of the subsequent packets, wherein the second flag value instructs nodes, of the load-balancing nodes, to use the send path list for routing the subsequent packets; and

means for forwarding the subsequent packets to the selected server on a route that is defined by the send path list and without hop-by-hop routing decisions.

22. A load-balancing router, comprising:

a network interface that is coupled to a data network for receiving one or more packet flows therefrom;

a processor;

one or more stored sequences of instructions for routing data from a client through one or more load-balancing nodes to a selected load-balanced server among a plurality of servers in a network and which, when executed by the processor, cause the processor to carry out the steps of;

receiving a first packet of a request from a client, wherein the load balancing router is a load balancing node in a path from the client to the plurality of servers;

storing a mapping of flow identifying information, associated with the first packet, to a client stickiness label and a send path list, wherein the sending path list identifies a path in the network from the client to a selected server;

storing an identifier of an egress interface on which the first packet arrives, in the sending path list, as the first packet of the client request is routed to the plurality of servers using hop-by-hop routing decisions; and

forwarding all subsequent packets associated with the client request to the selected server based on looking up the client stickiness label of such subsequent packets in the mapping.

23. A load-balancing router as recited in claim 22, wherein the one or more stored sequences of instructions cause the processor to carry out the further step of setting a flag in the first packet associated with the client request, wherein the flag indicates to nodes in the path from said client to the selected server that the nodes are to update the sending path list.

24. A load-balancing router as recited in claim 22, wherein the one or more stored sequences of instructions cause the processor to carry out the further step of setting a flag in a subsequent packet associated with the client request, wherein the flag instructs nodes to use the sending path list for routing the subsequent packet for server load balancing.

25. A load-balancing router, comprising:

a network interface that is coupled to a data network for receiving one or more packet flows therefrom;

a processor;

one or more stored sequences of instructions for routing data from a client through one or more load-balancing nodes to a selected load-balanced server among a plurality of servers in a network and which, when executed by the processor, cause the processor to carry out the steps of;

storing a mapping of flow identifying information, associated with the first IP packet, to a client stickiness label and a return path list, wherein the return path list identifies a path in the network from a selected server to the client, and wherein the return path list stores identifiers of egress interfaces;

receiving a server reply request packet, wherein the load-balancing router is a first load-balancing node in the network on a path from the client to the server;

extracting the client stickiness identifier from an IP header of the subsequent server reply packet;

looking up the client stickiness identifier value in the mapping; and

forwarding the subsequent server reply packet on an egress interface, of the egress interfaces in the return path list, that is determined based on a label in the mapping corresponding to the client stickiness identifier.

26. A load-balancing router, comprising:

a network interface that is coupled to a data network for receiving one or more packet flows therefrom;

a processor;

one or more stored sequences of instructions for routing data from a client through one or more load-balancing nodes to a selected load-balanced server among a plurality of servers in a network and which, when executed by the processor, cause the processor to carry out the steps of;

storing a mapping of flow identifying information, associated with the first IP packet, to a unique server identifier and a send path list, wherein the send path list identifies a path in the network from the client to the selected server, and wherein the send path list stores identifiers of egress interfaces on which the first IP packet arrives as the IP packet is routed to the selected server using hop-by-hop routing decisions;

receiving a subsequent client request packet, wherein the load balancing router is a last load-balancing node in the network on a path from the client to the selected server;

extracting the server identifier value from an IP header of the subsequent client request packet;

looking up the server identifier value in the mapping; and

forwarding the subsequent client request packet on an egress interface, of the egress interfaces in the send path list, that is determined based on a label in the mapping corresponding to the server identifier value.

27. A computer system for routing data from a client through one or more load-balancing routers to a selected load-balanced server among a plurality of servers in a network, the computer system comprising:

a first router node that is operable to receive a first packet of a request from a client, and operable to indicate to nodes in the path from said client to a selected server that the nodes are to update, based on hop-by-hop routing decisions, a send path list that identifies a path in the network from the client to a selected server;

a last router node associated with the plurality of servers, wherein the last router node is operable to store a client stickiness label in a mapping of client stickiness labels to server identifiers, and wherein the mapping associates the client stickiness label with a server identifier that uniquely identifies a selected server that has been selected from among the plurality of servers to receive the client request; and

an intermediate router node coupled communicatively between said first router node and said last router node, wherein the intermediate router is operable to update, based on a hop-by-hop routing decision, the send path list in response to receiving the first packet with the indication to update the send path list, and is operable to forward subsequent packets associated with the client request to the selected server on a route that is defined only by the send path list and without hop-by-hop routing decisions.

28. A computer system as recited in claim 27, wherein the intermediate router node is further operable to store one or more identifiers of egress interfaces on which the packet arrives, in the send path list, as the first packet of the client request is routed to the plurality of servers using hop-by-hop decisions.

29. A computer system as recited in claim 27, wherein the last router node is further operable to:

push the client stickiness label into the send path list, in response to receiving the first packet of the client requests;

receive one or more packets of a server response to the client request;

pop the client stickiness label from the send path list in response to receiving packets of the server response at the last node, and

forward the packets of the server response to the client based on the client stickiness label.

30. A computer system as recited in claim 27, wherein the first router node is further operable to set a first flag value in the first packet of the client content request.

31. A computer system as recited in claim 30, wherein the first flag value signals that the packet contains the send path list and comprises an IP-RECORD-ROUTE-OPTION-FOR-SLB flag value in a header of the first packet.

32. A computer system as recited in claim 27, wherein the first router node is further operable to set a first flag value in the first packet of the client request, wherein the first flag value signals that all nodes should update the send path list.

33. A computer system as recited in claim 27, wherein the first router node is further operable to set a second flag value in each of the subsequent packets.

34. A computer system as recited in claim 33, wherein the second flag value instructs nodes to use the send path list for routing the subsequent packets for server load balancing.

35. A computer system as recited in claim 33, wherein the second flag value instructs nodes to use the send path list for routing the subsequent packets and comprises an IP-STRICT-ROUTE-OPTION-FOR-SLB flag value in a header of the subsequent packets.

36. A computer system as recited in claim 27, wherein the first router node is further operable to select one of the servers to respond to the client request in response to receiving the first packet of the client request.

37. A computer system as recited in claim 27, wherein the client stickiness label uniquely identifies an association of the client to the selected server.

38. A computer system as recited in claim 27, wherein last router node is further operable to store the send path list in a mapping comprising an association of send path lists and client identifying information.

39. A computer system as recited in claim 27, wherein last router node is further operable to store the send path list in a mapping comprising an association of send path lists and packet flow identifying information.

40. A computer system as recited in claim 27, wherein the intermediate router node is further operable to store an identifier of an ingress interface on which the first packet of the client request arrives, in a return path list, as the first packet of the client request is routed from the first router node to the last router node using hop-by-hop decisions, based on setting an IP-RECORD-ROUTE-OPTION-FOR-SLB flag value in the first packet of the client request.

41. A computer system as recited in claim 40, wherein the last router node is further operable to:

receive one or more subsequent packets of a server reply; and

set an IP-STRICT-ROUTE-OPTION-FOR-SLB flag value in each of the subsequent packets of the server reply.

42. A computer system as recited in claim 41, wherein the intermediate router node is further operable to forward the subsequent packets of the server reply to the client on a route that is defined only by the return path list and without hop-by-hop routing decisions.

43. A computer system as recited in claim 42, wherein the first router node is further operable to store the return path list in a mapping comprising an association of return path lists and server identifiers.

44. A computer system as recited in claim 27, wherein the last router node is further operable to pop a server identifier from the send path list and forward the packet to the server on an IP address contained in the server identifier.

45. A computer system as recited in claim 27, wherein the last router node is further operable to pop a server identifier from the send path list, perform network address translation on the server identifier to determine where to forward the packet, and forward the packet to the server on an IP address determined by the network address translation.

46. A computer system as recited in claim 27, wherein the first router node and last router node are load-balancing routers that participate in the network as multicast nodes.