Distributed traffic controller for network data

• US 7,299,294 B1
• Filed: 04/12/2000
• Issued: 11/20/2007
• Est. Priority Date: 11/10/1999
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bruck, Jehoshua, LeMahieu, Paul, Fan, Chenggong, Bohossian, Vasken, Love, Philip
• Primary Examiner(s): Won; Michael
• Application Number:
  US09/548,188
• Time in Patent Office: 2,778 Days
• Field of Search: 709/222, 709/230, 709/232, 709/238, 709/244, 709/245, 709/251, 709/201, 709/217, 709/219, 709/227, 709/250, 709/225, 709/229, 709/234, 709/235, 709/241, 711/6, 711/203
• US Class Current: 709/235
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
• CPC Class Codes: H04L 67/1034 Reaction to server failures by a load balancer

Abstract

A distributed gateway for controlling computer network data traffic dynamically reconfigures traffic assignments among multiple gateway machines for increased network availability. If one of the distributed gateway machines becomes unavailable, traffic assignments are moved among the multiple machines such that network availability is substantially unchanged. The machines of the distributed gateway form a cluster and communicate with each other using a Group Membership protocol word such that automatic, dynamic traffic assignment reconfiguration occurs in response to machines being added and deleted from the cluster, with no loss in functionality for the gateway overall, in a process that is transparent to network users, thereby providing a distributed gateway functionality that is scalable. Operation of the distributed gateway remains consistent as machines are added and deleted from the cluster. A scalable, distributed, highly available, load balancing network gateway is thereby provided, having multiple machines that function as a front server layer between the network and a back-end server layer having multiple machines functioning as Web file servers, FTP servers, or other application servers. The front layer machines comprise a server cluster that performs fail-over and dynamic load balancing for both server layers.

Claims

1. A data traffic controller for a computer network, the data traffic controller comprising:

a network interface that permits communication with a subnet over which network data is sent and received; and

a plurality of computers connected in a cluster between the network interface and the subnet to form a distributed gateway for communicating network data between said network interface and said subnet, each of which computers executes a distributed gateway application that dynamically assigns any of a plurality of virtual network addresses selected from a pool of virtual addresses to any of a plurality of primary network addresses to send network data to an intended host on the subnet, wherein network data intended for said host on the subnet is addressed to one of the virtual network addresses, and wherein the computers communicate operational status information among themselves for each computer on the same subnet using a group membership protocol word to dynamically reconfigure network data traffic assignments among the plurality of computers in response to said status information for said dynamic assignment of virtual network addresses, wherein one computer can configure and monitor the operating condition of another computer in the distributed gateway cluster.

9. A data traffic controller as defined in claim 1, wherein the operational status includes the network data traffic loads being carried by each computer.

10. A data traffic controller as defined in claim 9, wherein a computer communicates with the other computers on the same subnet to form said distributed gateway.

11. A data traffic controller as defined in claim 10, wherein the computers dynamically reconfigure the address assignments among the distributed gateway computers in response to traffic loads being carried by the computers.

12. A data traffic controller as defined in claim 11, wherein a computer determines that it is carrying a traffic load that is less than the traffic load being carried by one of the other computers of said plurality, and in response reconfigures the address assignments to assume a portion of the traffic load being handled by the one other computer and thereby reduces the traffic load of the one other computer.

13. A data traffic controller as defined in claim 11, wherein the traffic load being carried by a computer of the distributed gateway cluster increases when another one of the computers in the distributed gateway cluster becomes unavailable, and in response the the computers reconfigure the traffic load being handled by the computers.

14. A data traffic controller as defined in claim 11, wherein operations of the data traffic controller are substantially unchanged after a reconfiguration of the address assignments, thereby making the reconfiguration transparent to a user.

15. A data traffic controller as defined in claim 14, wherein the operations of the data traffic controller include one or more computer application programs.

16. A data traffic controller as defined in claim 15, wherein the computer application program comprises a firewall program.

17. A data traffic controller as defined in claim 15, wherein the computer application program comprises a mail server program.

18. A data traffic controller as defined in claim 15, wherein the computer application program comprises a file server program.

19. A data traffic controller as defined in claim 10, wherein upon a new computer communicating with the plurality of computers of the distributed gateway cluster of computers already communicating among themselves, the new computer joins the distributed gateway cluster.

20. A data traffic controller as defined in claim 19, wherein the computers of the distributed gateway cluster recognize that the new computer has a lower traffic load than the remaining computers of the distributed gateway cluster when the new computer joins the cluster, and in response the new computer assumes a portion of the traffic load from one or more of the computers in a process that dynamically reconfigures the traffic address assignments of the computers.

21. A data traffic controller as defined in claim 19, wherein the computers of the cluster detect when the new computer begins communicating with the cluster, thereby joining the cluster, and in response the computers perform a dynamic reconfiguration process to change one or more traffic address assignments from the computers to the new computer.

22. A data traffic controller as defined in claim 1, wherein a computer communicates by passing said group membership protocol word to the other computers along the subnet.

23. A data traffic controller as defined in claim 22, wherein the group membership protocol word includes node map configuration data indicating traffic assignments for each one of the computers in the distributed gateway cluster.

24. A data traffic controller as defined in claim 22, wherein the group membership protocol word includes data that indicates the traffic load being carried by each computer in the distributed gateway cluster.

25. A data traffic controller as defined in claim 9, wherein a computer can receive a user input that specifies a configuration or monitoring change for another computer, and can receive the user input from a remote user.

26. A data traffic controller as defined in claim 10, wherein the data traffic controller performs the dynamic reconfiguration of address assignments through a graphical user interface (GUI).

27. A data traffic controller as defined in claim 26, wherein subnet virtual network addresses and traffic address assignments are graphically represented, and the reconfiguration of address assignments can be performed by a drag and drop action of traffic address assignments onto subnet virtual network addresses.

28. A data traffic controller as defined in claim 26, wherein the GUI includes a local status monitor indicator for the network interface.

29. A data traffic controller as defined in claim 26, wherein the GUI includes a local condition monitor indicator for the application program of the data traffic controller.

30. A data traffic controller as defined in claim 26, wherein the GUI includes a local condition monitor indicator for a ping communication utility.

2. A method of controlling data traffic for a computer network through a cluster of a plurality of computers connected as a distributed gateway between a network interface and a subnet over which network data is sent and received, the method comprising:

receiving network data intended for a host on the subnet at said plurality of computers, wherein the network data is addressed to one of a plurality of virtual network addresses selected from a pool of virtual network addresses that are on the subnet and are associated with one or more primary network addresses;

communicating by a computer with one or more of said plurality of computers on the same subnet operational status information about said one or more computers; and

dynamically assigning by said plurality of computers in response to said status information said one virtual network address to a primary network address to send the network data to the intended host on the subnet, wherein a computer can configure and monitor the operating condition of another computer in the distributed gateway cluster.

3. A method of controlling data traffic as defined in claim 2, wherein said computers comprise server computers that are all members of a first subnet of network addresses over which network data is sent and received, and further comprising:

communicating with the plurality of server computers state information and load information with a dynamic reconfiguration protocol that permits reassignment of network addresses among the server computers in response to said state information and said load information, said status information comprising said state information and said load information; and

communicating between said server computers and a plurality of host computers that are members of a host subnet of network addresses to send and receive network data traffic.

4. A method as defined in claim 3, wherein communicating with said plurality of server computers comprises sending data using a Reliable Message layer scheme that comprises a token data packet and one or more data carriage packets, wherein the token data packet specifies the number of data carriage packets that together comprise a Reliable Message packet and wherein the data carriage packets include data relating to state information and data traffic load information about each of the server computers.

5. A method as defined in claim 3, further including:

configuring an operating system of the server computers such that all network addresses in a pool of addresses assigned to the first subnet are assigned to the server computers;

generating a gratuitous address resolution protocol (ARP) message in response to an address reassignment of one server computer and communicating the ARP message to the other server computers of the first subnet;

blocking the sending of an ARP acknowledgment message to the other server computers of the first subnet for any received gratuitous ARP message, thereby inhibiting reboot operation of the respective server computers and ensuring that each server computer is unaware of any duplicate assignment of network address numbers.

6. A method as defined in claim 3, further including operating as an authoritative node of the first subnet to ensure symmetric routing of network data traffic to and from the first subnet.

7. A method as defined in claim 6, wherein operating to ensure symmetric traffic routing comprises:

receiving a data request from a responding server computer of the first subnet, wherein the data request was initially received at the responding server computer, which determined the authoritative node for responding to the data request;

identifying a server computer in the first subnet that will handle the data traffic associated with the data request and forwarding the data request to the identified server computer for handling;

receiving a reply message from a server computer of the first subnet that is operating as a default reply node to a second subnet computer that is responding to the data request; and

forwarding the reply message to a server computer of the first subnet that will ensure symmetric routing of the data request and reply message with respect to the server computers of the first subnet.

8. A method as defined in claim 7, further comprising forwarding assignment information to the server computer of the first subnet that was operating as the default reply node for the data request, wherein the assignment information includes forwarding information that the default reply node can use to directly forward response messages from the second subnet computer to the first subnet computer that will ensure symmetric routing.

31. A method as defined in claim 2, wherein the operational status includes the network data traffic loads being carried by each computer.

32. A method as defined in claim 31, wherein each computer communicates with the other computers on the same subnet and thereby forms a distributed gateway cluster of computers.

33. A method as defined in claim 32, further including the computers dynamically reconfiguring the address assignments among the distributed gateway cluster computers in response to traffic loads being carried by the computers.

34. A method as defined in claim 33, wherein reconfiguring comprises a computer determining that it is carrying a traffic load that is less than the traffic load being carried by one of the other computers, and in response reconfiguring the address assignments to assume a portion of the traffic load being handled by another computer and thereby reducing the traffic load of the other computer.

35. A method as defined in claim 33, wherein the traffic load being carried by the other computer increases when one of the remaining computers in the distributed gateway cluster becomes unavailable, and in response the computers reconfigure the traffic load being handled by the computers.

36. A method as defined in claim 33, wherein reconfiguring the address assignments results in substantially no change in the operations of the data traffic controller, thereby making the reconfiguration transparent to a user.

37. A method as defined in claim 36, wherein the operations of the data traffic controller include one or more computer application programs.

38. A method as defined in claim 37, wherein the computer application program comprises a firewall program.

39. A method as defined in claim 37, wherein the computer application program comprises a mail server program.

40. A method as defined in claim 37, wherein the computer application program comprises a file server program.

41. A method as defined in claim 32, wherein upon a new computer communicating with a computer of the distributed gateway cluster of computers already communicating among themselves, said new computer joins the distributed gateway cluster.

42. A method as defined in claim 41, wherein the computers of the distributed gateway cluster recognize that a joining computer has a lower traffic load than the computers of the distributed gateway cluster when the computer joins the cluster, and in response the joining computer assumes a portion of the traffic load from one or more of the computers in a process that dynamically reconfigures the traffic address assignments of the cluster.

43. A method as defined in claim 41, wherein upon detecting when a new computer begins communicating with the cluster, thereby joining the cluster, performing a dynamic reconfiguration process to change one or more traffic address assignments from the computers of the cluster to the new computer.

44. A method as defined in claim 41, wherein said communicating comprises passing a group membership protocol word to the computers of the subnet.

45. A method as defined in claim 44, wherein the group membership protocol word includes node map configuration data indicating traffic assignments for each one of the computers in the distributed gateway cluster.

46. A method as defined in claim 44, wherein the group membership protocol word includes data that indicates the traffic load being carried by each computer in the distributed gateway cluster.

47. A method as defined in claim 31, wherein a computer can receive a user input that specifies a configuration or monitoring change for another computer, and can receive the user input from a remote user.

48. A method as defined in claim 32, wherein said dynamically assigning comprises dynamic reconfiguration of address assignments through a graphical user interface (GUI).

49. A method as defined in claim 48, wherein the subnet virtual network addresses and traffic address assignments are graphically represented, and the reconfiguration of address assignments comprises a drag and drop action of traffic address assignments onto subnet virtual network addresses.

50. A method as defined in claim 48, wherein the GUI includes a local status monitor indicator for the network interface.

51. A method as defined in claim 48, wherein the GUI includes a local condition monitor indicator for the application program of the data traffic controller.

52. A method as defined in claim 48, wherein the GUI includes a local condition monitor indicator for a ping communication utility.

53. A program product for use in a computer of a plurality of computers connected in a cluster as a distributed gateway between a network interface and a subnet of a computer network over which network data is sent and received, wherein the network data is received through the network interface and communicated through the computer cluster to the subnet, the program product comprising:

a recordable medium embodying computer-readable instructions executable by the computers to perform a method comprising;

receiving by said plurality of computers network data intended for a host on the subnet, wherein the network data is addressed to one of a plurality of virtual network addresses selected from a pool of virtual network addresses that are on the subnet and are associated with one or more primary network addresses;

communicating among one or more of said plurality of computers on the same subnet operational status information about said computers; and

dynamically assigning by said plurality of computers of said cluster in response to said communicating said status information said virtual network address to a primary network address to send the network data to the intended host on the subnet, thereby controlling network data traffic, wherein a computer can configure and monitor the operating condition of another computer in the distributed gateway cluster.

54. A program product as defined in claim 53, wherein the operational status includes the network data traffic loads being carried by each computer.

55. A program product as defined in claim 54, wherein a computer communicates with the other computers on the same subnet and thereby forms a distributed gateway cluster of computers.

56. A program product as defined in claim 55, further including dynamically reconfiguring the address assignments among the distributed gateway cluster computers in response to traffic loads being carried by the computers.

57. A program product as defined in claim 56, wherein said reconfiguring comprises a computer determining that it is carrying a traffic load that is less than the traffic load being carried by one of the other computers, and in response reconfiguring the address assignments to assume a portion of the traffic load being handled by the other computers and thereby reducing the traffic load of the other computers.

58. A program product as defined in claim 56, wherein the traffic load being carried by the other computer increases when one of the other computers in the distributed gateway cluster becomes unavailable, and in response reconfiguring the traffic load being handled by the other computers.

59. A program product as defined in claim 56, wherein operations are substantially unchanged after a reconfiguration of the address assignments, thereby making the reconfiguration transparent to a user.

60. A program product as defined in claim 58, wherein the operations of the data traffic controller include one or more computer application programs.

61. A program product as defined in claim 60, wherein the computer application program comprises a firewall program.

62. A program product as defined in claim 60, wherein the computer application program comprises a mail server program.

63. A program product as defined in claim 60, wherein the computer application program comprises a file server program.

64. A program product as defined in claim 55, wherein a computer begins communicating with a distributed gateway cluster of computers already communicating among themselves, thereby joining the distributed gateway cluster.

65. A program product as defined in claim 64, wherein the computers of the distributed gateway cluster recognize that a computer has a lower traffic load than the remaining computers of the distributed gateway cluster when the computer joins the cluster, and in response the computer assumes a portion of the traffic load from one or more of the computers in a process that dynamically reconfigures the traffic address assignments of the cluster.

66. A program product as defined in claim 64, wherein the computers detect when a new computer begins communicating with the cluster, thereby joining the cluster, and in response the computers perform a dynamic reconfiguration process to change one or more traffic address assignments to the new computer.

67. A program product as defined in claim 54, wherein a computer communicates by passing a group membership protocol word to the other computers along the subnet.

68. A program product as defined in claim 67, wherein the group membership protocol word includes node map configuration data indicating traffic assignments for each one of the computers in the distributed gateway cluster.

69. A program product as defined in claim 67, wherein the group membership protocol word includes data that indicates the traffic load being carried by each computer in the distributed gateway cluster.

70. A program product as defined in claim 54, wherein a computer can receive a user input that specifies a configuration or monitoring change for another computer, and can receive the user input from a remote user.

71. A program product as defined in claim 55, wherein the data traffic controller performs the dynamic reconfiguration of address assignments through a graphical user interface (GUI).

72. A program product as defined in claim 71, wherein the subnet virtual network addresses and traffic address assignments are graphically represented, and the reconfiguration of address assignments can be performed by a drag and drop action of traffic address assignments onto subnet virtual network addresses.

73. A program product as defined in claim 71, wherein the GUI includes a local status monitor indicator for the network interface.

74. A program product as defined in claim 71, wherein the GUI includes a local condition monitor indicator for the application program of the data traffic controller.

75. A program product as defined in claim 71, wherein the GUI includes a local condition monitor indicator for a ping communication utility.