DISTRIBUTED LOAD BALANCING SYSTEMS

• US 20160094452A1
• Filed: 12/01/2014
• Published: 03/31/2016
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Jain, Jayant, Sengupta, Anirban, Parthasarathy, Mohan, Sequeira, Allwyn, Maskalik, Serge, Lund, Rick
• Granted Patent: US 10,135,737 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 2101/622 Layer-2 addresses, e.g. medium access control [MAC] addresses
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 45/44 Distributed routing
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 61/2521 Translation architectures other than single NAT servers
• CPC Class Codes: H04L 61/5069 for group communication, multicast communication or broadcast communication
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1025 Dynamic adaptation of the criteria on which the server selection is based
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

Some embodiments provide a novel method for load balancing data messages that are sent by a source compute node (SCN) to one or more different groups of destination compute nodes (DCNs). In some embodiments, the method deploys a load balancer in the source compute node'"'"'s egress datapath. This load balancer receives each data message sent from the source compute node, and determines whether the data message is addressed to one of the DCN groups for which the load balancer spreads the data traffic to balance the load across (e.g., data traffic directed to) the DCNs in the group. When the received data message is not addressed to one of the load balanced DCN groups, the load balancer forwards the received data message to its addressed destination. On the other hand, when the received data message is addressed to one of load balancer'"'"'s DCN groups, the load balancer identifies a DCN in the addressed DCN group that should receive the data message, and directs the data message to the identified DCN. To direct the data message to the identified DCN, the load balancer in some embodiments changes the destination address (e.g., the destination IP address, destination port, destination MAC address, etc.) in the data message from the address of the identified DCN group to the address (e.g., the destination IP address) of the identified DCN.

Claims

1. A distributed load balancing (DLB) system comprising:

a plurality of host computing devices on which a plurality of source compute nodes (SCNs) execute;

at least one load balancer executing on each host computing device, each load balancer for intercepting data messages sent by a set of SCN on the load balancer'"'"'s host computing device, and distributing the data messages that are addressed to a destination compute node (DCN) group to the DCNs in the group; and

a set of controllers for configuring the load balancers on the host computing device.

2. The DLB system of claim 1, wherein the set of controllers includes at least controller for deploying at least one SCN on the host computing devices.

5. The DLB system of claim 1, wherein the set of controllers configures the load balancers by distributing load balancing criteria to the load balancers.

6. The DLB system of claim 5, wherein the set of controllers further configures the load balancers by collecting message traffic data from the load balancers, adjusting the load balancing criteria, and distributing adjusted load balancing criteria to the load balancers.

7. The DLB system of claim 6, wherein the message traffic data from a load balancer comprises data regarding the number of data messages that the load balancer forwarded to each DCN in the DCN group.

8. The DLB system of claim 6, wherein the message traffic data from a load balancer comprises data regarding the number of data message flows that the load balancer forwarded to each DCN in the DCN group.

9. The DLB system of claim 1,

wherein each load balancer spreads the data messages among the DCNs based on a set of load balancing criteria,

wherein the set of controllers configures the load balancers by collecting message traffic data from the load balancers, adjusting the load balancing criteria, and distributing adjusted load balancing criteria to the load balancers.

10. The DLB system of claim 9, wherein the distributed load balancing criteria specifies numerical values that affect how the load balancer spreads the data messages among the DCN.

11. The DLB of claim 10, wherein the numerical values include one weight value for each DCN, wherein based on the weight values, the load balancer performs a weighted round robin selection of the DCNs for new data message flows.

12. The DLB system of claim 1, wherein the set of controllers configures the load balancers by distributing addresses of the DCNs that are part of the DCN group.

13. The DLB system of claim 1, wherein the set of controllers configures the load balancers by (i) distributing a list of the DCNs that are part of the DCN group, and (2) when a DCN is to be added or removed from the DCN group, distributing an updated list that adds or removes the added or removed DCN from the DCN group.

14. The DLB system of claim 1,

wherein each load balancer spreads the data messages among the DCNs based on a set of load balancing criteria,

wherein the set of controllers is further for collecting message traffic data from the load balancers, generating aggregated message traffic data from the collected message traffic data, and distributing the aggregated message traffic data to the load balancers;

wherein based on the distributed aggregated message traffic data, each load balancer adjusts the set of load balancing criteria that the load balancer uses to spread the data messages among the DCNs.

15. The DLB system of claim 14, wherein the load balancing criteria specifies numerical values that control how the load balancer spreads the data messages among the DCN.

16. The DLB system of claim 1, wherein the set of controllers includes more than one controller.

17. The DLB system of claim 1, wherein the set of controllers includes only one controller.

18. The DLB system of claim 1, wherein

each load balancer is for processing data messages from one SCN on one host computing device,

the controller set is further for deploying the SCNs on the host computing device, and

the controller set configures each load balancer as part of the deployment of the load balancer'"'"'s SCN.

19. The DLB system of claim 1,

wherein each load balancer spreads the data messages among the DCNs by changing a destination address in each data message from an address of the identified DCN group to an address of one of the DCN;

and

wherein each address is one of an Internet Protocol (IP) address, a media access control (MAC) address, and an Internet Protocol (IP) port address.

20. The DLB system of claim 1, wherein after changing the destination address of a data message, the load balancer supplies the data message to a software forwarding element (SFE) that executes on the host computing device with the load balancer, in order to have the SFE forward the data message to the identified DCN.

21. The DLB system of claim 20, wherein the SFE forwards the data message to the identified DCN either (i) through a set of intermediate forwarding elements that operate outside of the host computing device when the DCN is outside of the SFE'"'"'s host computing device, or (ii) directly to the DCN when the DCN executes on the SFE'"'"'s host computing device.

22. The DLB system of claim 1, wherein the SCNs are not configured to send the data messages to the load balancers, and thereby do not address the data messages to the load balancers.

23. The DLB system of claim 1, wherein

the DCN group is a first DCN group, and

at least one load balancer is further for intercepting, on the load balancer'"'"'s host computing device, data messages that are sent by at least one SCN to a second DCN group and spreading the data messages among the DCNs of the second group.

24. The DLB system of claim 1, wherein

the DCNs are data compute end nodes or middle box service nodes, and

the SCNs are virtual machines.

25. The DLB system of claim 1, wherein

each load balancer is for processing data messages from one SCN on one host computing device;

and

each load balancer enforces the load balancing rules needed to spread the data traffic from the load balancer'"'"'s associated SCN.

26. The DLB system of claim 1, wherein

each load balancer is for processing data messages from a set of one or more SCNs on one host computing device;

and

each load balancer enforces the load balancing rules needed to spread the data traffic from the load balancer'"'"'s associated SCN set.

3. The DLB of system of claim 3,

wherein the set of controllers configures the load balancers by distributing data regarding data message traffic received at different DCNs in the DCN group;

and

wherein the distributed message traffic data comprises statistic data that the controller set gathers from the load balancers.

4. The DLB of system of claim 3 further comprising a load balancing agent executing on each host computing device to compute load balancing criteria for at least one load balancer executing on the host based on the distributed message traffic data, wherein each load balancer distributes the data messages among the DCNs in the DCN group based on the load balancing criteria.