Transparent migration of TCP based connections within a network load balancing system

• US 8,244,864 B1
• Filed: 03/20/2001
• Issued: 08/14/2012
• Est. Priority Date: 03/20/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bahl, Pradeep, Joy, Joseph M., Sun, Feng
• Primary Examiner(s): Bengzon, Greg C
• Application Number:
  US09/812,931
• Time in Patent Office: 4,165 Days
• Field of Search: 709/238, 709/239, 709226-228, 709/225, 718/105, 395/674
• US Class Current: 709/225
• CPC Class Codes: H04L 67/148 Migration or transfer of sessions
• CPC Class Codes: H04L 69/162 involving adaptations of sockets based mechanisms secure socket layer H04L63/168

Abstract

A method and system for enhancing a network load balancing system'"'"'s ability to distribute connection information and data amongst the plurality of nodes within the system is presented. A client connection request is redirected to a load balancing service residing on a receiving node by modifying the destination port information contained within the packet. The receiving node can be any node within the load balancing system, and is determined from amongst the plurality of nodes that comprise the system by a conventional means. A transparent connection is then established between the client and the load balancing service residing on the node by modifying subsequently transmitted connection setup packets sent by the receiving node and the client. A target node is determined according to resource specific information contained within initially received data packets from the client. The target node determined this way could be the local node or another node. If it is another node, the connection parameters and the data received over the connection are passed to a peer load balancing service residing on the target node. The target load balancing service then instructs TCP/IP to create a Transmission Control Block for the connection and TCP/IP indicates the connection to the appropriate client resource. The target load balancing service sends a “migration completed” message to the receiving node. Upon receiving this notification, the receiving load balancing service instructs a corresponding load balancing driver to stop handling subsequent packets received from that particular client, while the target node is enabled to accept the packets.

Claims

1. A method for distributing connection information and data related to a client session within a load balancing system having a plurality of nodes, the distribution being made according to resource specific information contained within the initial data packets transmitted by the client, the method comprising:

directly receiving a client connection request packet by any one and only one of the plurality of nodes in the system of load balanced nodes without a centralized load balancing node, each of nodes in the system having an interceptor and a load balancing service residing on each respective node, wherein subsequent client connection requests are received by respective ones and only ones of other nodes of the plurality of nodes;

intercepting the client connection request packet by an interceptor residing on a receiving node determined from amongst the plurality of nodes;

changing a destination port number specified in the intercepted client connection request packet to an assigned port number of a load balancing service residing on the receiving node;

redirecting the packet to the load balancing service on the receiving node using the assigned port number;

accepting the client connection request by the load balancing service residing on the receiving node;

determining a target node to which the connection information and data will be migrated to; and

migrating the connection information and data to the target node.

2. A method as in claim 1, wherein the packet is a TCP SYN packet.

3. A method as in claim 2, wherein the interceptor changes the source port number of subsequently transmitted connection setup packets and ACK packets for the one or more data packets sent by the client from the assigned port number of the load balancing service to the original destination port number before the packets are transmitted to the client.

4. A method as in claim 2, wherein the interceptor changes the destination port number of subsequently transmitted ACK packets sent by the client from the assigned port number of the client specified application to the destination port number of the load balancing service residing on the receiving node.

5. A method as in claim 1, wherein the step of accepting includes establishing a connection between the client and a load balancing service residing on the receiving node.

6. A method as in claim 1, wherein the step of accepting includes allocating a TCP control block and connection descriptor for the connection that specify the destination port number as the assigned port number of the load balancing service residing on the receiving node.

7. A method as in claim 1, wherein the step of accepting further includes receiving at least an initial data packet from the client upon establishment of the connection, the data packet containing the resource specific information.

8. A method as in claim 1, wherein the step of accepting further includes suppressing subsequently transmitted client data packets to minimize the amount of data to be migrated to the target node.

9. A method as in claim 1, wherein the step of determining includes hashing over the resource specific information contained within at least one client data packet.

10. A method as in claim 9, wherein the step of hashing is performed by the receiving node.

11. A method as in claim 1, wherein the step of migrating includes establishing a secure connection through a control channel between the receiving node and the target node when the receiving node is not determined as the target node.

12. A method as in claim 11, wherein the control channel has a different port number than the assigned port number of the load balancing service residing on the target node.

13. A method as in claim 1, wherein the step of migrating includes transferring the connection information and data from the receiving node to a load balancing service residing on the target node when the receiving node is not determined as the target node.

14. A method as in claim 13, wherein the connection information and data is used by the target node to establish a connection between the client and the resource residing on the target node as specified initially by the client.

15. A method as in claim 1, wherein the step of migrating includes servicing the connection by the receiving node when the receiving node is determined as the target node.

16. A method as in claim 1, wherein the step of migrating further includes passing the client data to the resource residing on the target node as specified by the client.

17. A method as in claim 1, further comprising the step of processing any subsequently received packets sent during the same session by the same target node.

18. A hardware computer readable storage medium having further computer executable instructions for performing the steps of:

receiving a client connection request packet by any one and only one of the plurality of nodes in a system of load balanced nodes without a centralized load balancing node, each of nodes in the system having an interceptor and a load balancing service residing on each respective node, wherein subsequent client connection requests are received by respective ones and only ones of other of the plurality of nodes;

intercepting the client connection request packet by an interceptor residing on a receiving node determined from amongst the plurality of nodes;

changing a destination port number specified in the intercepted client connection request packet to an assigned port number of a load balancing service residing on the receiving node;

redirecting the packet to the load balancing service using the assigned port number;

accepting the client connection request by the load balancing service residing on the receiving node;

determining a target node to which the connection information and data will be migrated to; and

migrating the connection information and data to the target node.

19. A method for migrating connection information and data related to a client session between nodes within a load balancing system having a plurality of nodes, the method comprising the steps of:

receiving a client connection request packet by any one and only one of the plurality of nodes in the system of load balanced nodes, each of nodes in the system having an interceptor and a load balancing service residing on each respective node, wherein subsequent client connection requests are received by respective ones of other of the plurality of nodes;

intercepting the client connection request packet by an interceptor residing on a receiving node determined from amongst the plurality of nodes;

changing a destination port number specified in the intercepted client connection request packet to an assigned port number of a load balancing service residing on the receiving node;

redirecting the packet to the load balancing service;

determining a target node that the connection information and data will be migrated to;

migrating the connection information and data to another load balancing service residing on the target node; and

establishing a connection between the client and the target node;

wherein the step of determining includes hashing over resource specific information contained within the client connection request packet sent during the client session by only the receiving node.

20. A method as in claim 19, wherein the packet is a client TCP SYN.

21. A method as in claim 20, wherein the interceptor changes the source port number of subsequently transmitted connection setup packets and ACK packets for the one or more data packets sent by the client from the assigned port number of the load balancing service to the original destination port number before the packets are transmitted to the client.

22. A method as in claim 20, wherein the interceptor changes the destination port number of subsequently transmitted ACK packets sent by the client from the assigned port number of the client specified application to the destination port number of the load balancing service residing on the receiving node.

23. A method as in claim 19, wherein the step of redirecting further includes establishing a session between the client and the load balancing service residing on a receiving node.

24. A method as in claim 19, wherein the step of hashing is performed by the same node within the load balancing system that accepted the client connection request.

25. A method as in claim 19, wherein the step of migrating includes querying TCP/IP at the receiving node to obtain the connection information and associated data.

26. A method as in claim 19, wherein the step of migrating includes establishing a secure connection through a control channel between the receiving node and the target node.

27. A method as in claim 26, wherein the control channel has a different port number than the assigned port number of the load balancing service residing on the target node.

28. A method as in claim 19, wherein the step of migrating includes transferring the connection information and data from the receiving node to a load balancing service residing on the target node.

29. A method as in claim 19, wherein the step of migrating further includes passing the connection information to the TCP/IP stack of the target node.

30. A method as in claim 19, wherein the step of establishing comprises:

allocating a TCP control block, the TCP control block specifying the destination port number as that indicated initially by the client;

notifying the application and socket corresponding to the specified destination port of the connection;

wherein acceptance of the connection by the application is indicated by returning a SUCCESS signal and passing the client data to TCP/IP to be indicated to the specified resource;

wherein rejection of the connection results in TCP/IP on the target node communicating FAILURE to the target load balancing service, such that the target load balancing service transmits a NACK signal to the load balancing service residing on the receiving node;

sending a migration complete ACK signal from the target load balancing service to the load balancing service residing on the receiving node upon passage of the data to the specified target resource to indicate the successful migration of the connection information and data;

receiving an ACK from the load balancing service residing on the receiving node in response to the migration complete ACK signal sent by the target load balancing service; and

sending an instruction from the target load balancing service to a target load balancing driver indicating that the target load balancing driver can start processing packets from the client whose connection was successfully migrated.

31. A method as in claim 19 further including the step of instructing the load balancing driver on the receiving node to stop accepting packets on that connection from the client, and instructing TCP/IP on the receiving node to delete the TCP control block and connection descriptor associated with migrated connection upon receiving an ACK signal from the target node.

32. A method as in claim 19 further including the step of terminating the client connection upon receiving a NACK signal from the target load balancing service, the connection being terminated by the load balancing service residing on the receiving node.

33. A method as in claim 32, wherein the client connection is terminated by closing the socket for the connection and transmitting a TCP RST packet to the client.

34. A method as in claim 33, wherein a source port data field of the TCP RST packet is modified by the interceptor to indicate the port number of the client specified resource.

35. A method as in claim 19, further comprising the step of processing any subsequently received packets sent during the same session by the same target node.