Dynamically adjusting a data compute node group

• US 10,341,233 B2
• Filed: 07/31/2015
• Issued: 07/02/2019
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Jain, Jayant, Sengupta, Anirban, Parthasarathy, Mohan, Sequeira, Allwyn, Maskalik, Serge, Lund, Rick
• Primary Examiner(s): Tran, Jimmy H
• Application Number:
  US14/815,838
• Publication Number:
  US 20160094631A1
• Time in Patent Office: 1,432 Days
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 2101/622 Layer-2 addresses, e.g. medium access control [MAC] addresses
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 45/44 Distributed routing
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 61/2521 Translation architectures other than single NAT servers
• CPC Class Codes: H04L 61/5069 for group communication, multicast communication or broadcast communication
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1025 Dynamic adaptation of the criteria on which the server selection is based
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

Some embodiments provide a novel method for load balancing data messages that are sent by a source compute node (SCN) to one or more different groups of destination compute nodes (DCNs). In some embodiments, the method deploys a load balancer in the source compute node'"'"'s egress datapath. This load balancer receives each data message sent from the source compute node, and determines whether the data message is addressed to one of the DCN groups for which the load balancer spreads the data traffic to balance the load across (e.g., data traffic directed to) the DCNs in the group. When the received data message is not addressed to one of the load balanced DCN groups, the load balancer forwards the received data message to its addressed destination. On the other hand, when the received data message is addressed to one of load balancer'"'"'s DCN groups, the load balancer identifies a DCN in the addressed DCN group that should receive the data message, and directs the data message to the identified DCN. To direct the data message to the identified DCN, the load balancer in some embodiments changes the destination address (e.g., the destination IP address, destination port, destination MAC address, etc.) in the data message from the address of the identified DCN group to the address (e.g., the destination IP address) of the identified DCN.

Claims

1. A non-transitory machine readable medium storing a program for managing a set of middlebox service destination compute nodes (DCNs) that process data messages from a set of source compute nodes (SCNs), the program comprising sets of instructions for:

collecting, from a plurality of load balancers executing on a plurality of host computers along with the set of SCNs and implementing a distributed load balancer for the set of middlebox service DCNs, message traffic load statistics relating to data messages from the SCNs that the plurality of load balancers distribute to the DCNs of the DCN set, the data messages from the SCNs being addressed to data compute end nodes (DCENs) other than the middlebox service DCN set before being redirected to the middlebox service DCN set, each middlebox service DCN in the middlebox service DCN set performing a middlebox service operation on SCN data messages distributed to the particular middlebox service DCN from the distributed load balancer;

analyzing the message traffic load statistics; and

based on the analyzed message traffic load statistics, directing a set of one or more controllers to modify the middlebox service DCN set'"'"'s membership.

2. The non-transitory machine readable medium of claim 1, wherein the set of instructions for directing the controller set comprises a set of instructions for directing the controller set to remove a middlebox service DCN from the middlebox service DCN set when the analyzed message traffic load data indicates that one or more middlebox service DCNs in the middlebox service DCN set are being underutilized.

3. The non-transitory machine readable medium of claim 1, wherein the set of instructions for directing the controller set comprises a set of instructions for directing the controller set to add a new middlebox service DCN to the middlebox service DCN set when the data message load on the middlebox service DCN set or on one middlebox service DCN in the middlebox service DCN set exceeds a threshold value.

4. The non-transitory machine readable medium of claim 3, wherein the controller set adds a new middlebox service DCN by provisioning a new middlebox service DCN and adding the new middlebox service DCN to the middlebox service DCN set.

5. The non-transitory machine readable medium of claim 3, wherein the controller set adds a new middlebox service DCN by allocating a previously provisioned middlebox service DCN to the middlebox service DCN set.

6. The non-transitory machine readable medium of claim 1, wherein the load balancers are deployed in the egress datapaths of the SCNs and distribute the data messages from the SCNs to the middlebox service DCNs in the middlebox service DCN set.

7. The non-transitory machine readable medium of claim 6, wherein at least one load balancer for at least one group of SCNs (i) executes on one host computer with the group of SCNs, and (ii) intercepts data messages sent from the SCNs in the SCN group and directs in a load balanced manner the intercepted data messages to the middlebox service DCNs of the middlebox service DCN set when data messages need to be processed by the middlebox service DCN set.

8. The non-transitory machine readable medium of claim 7, wherein the SCN group includes only one SCN.

9. The non-transitory machine readable medium of claim 7, wherein the SCN group includes more than one SCN.

10. The non-transitory machine readable medium of claim 7, wherein at least one intercepted data message is directed to a middlebox service DCN in the middlebox service DCN set that executes on a host computer different than the host computer on which the load balancer executes.

11. The non-transitory machine readable medium of claim 6,

wherein a plurality of SCNs and a plurality of load balancers execute on a set of host computers;

wherein each load balancer in the plurality of load balancers is associated with one SCN, executes on the same host computer at its associated SCN, intercepts data messages sent from its associated SCN and directs in a load balanced manner the intercepted data messages to the middlebox service DCNs of the middlebox service DCN set when data messages need to be processed by the middlebox service DCN set.

12. The non-transitory machine readable medium of claim 1, wherein the set of instructions for directing the set of one or more controllers to modify the middlebox service DCN set'"'"'s membership comprises automatically directing the controller to modify the middlebox service DCN set'"'"'s membership without receiving any user input.

13. The non-transitory machine readable medium of claim 1, wherein the middlebox service DCNs execute on host computers in the same datacenter as the SCNs.

14. The non-transitory machine readable medium of claim 1, wherein the middlebox service provided by the middlebox service DCN set is at least one of a firewall service, an intrusion detection service, and a wide area network optimization service.

15. The non-transitory machine readable medium of claim 1, wherein a load balancer performs an address translation operation that translates an address of a DCEN to an address of a middlebox service DCN in the middlebox service DCN set to distribute the data message to the middlebox service DCN in the middlebox service DCN set.

16. The non-transitory machine readable medium of claim 1, wherein, after performing a middlebox service operation, a middlebox service DCN in the middlebox service DCN set forwards the processed SCN data message to the DCEN.

17. A method for managing a set of middlebox service destination compute nodes (DCNs) that process data messages from a set of source compute nodes (SCNs), the method comprising:

collecting, from a plurality of load balancers executing on a plurality of host computers along with the set of SCNs and implementing a distributed load balancer for the set of middlebox service DCNs, message traffic load statistics relating to data messages from the SCNs that the plurality of load balancers distribute to the DCNs of the DCN set, the data messages from the SCNs being addressed to data compute end nodes (DCENs) other than the middlebox service DCN set before being redirected to the middlebox service DCN set, each middlebox service DCN in the middlebox service DCN set performing a middlebox service operation on SCN data messages distributed to the particular middlebox service DCN from the distributed load balancer;

analyzing the message traffic load statistics; and

based on the analyzed message traffic load statistics, directing a set of one or more controllers to modify the middlebox service DCN set'"'"'s membership.

18. The method of claim 17, wherein directing the controller set comprises directing the controller set to remove a middlebox service DCN from the middlebox service DCN set when the analyzed message traffic load data indicates that one or more middlebox service DCNs in the middlebox service DCN set are being underutilized.

19. The method of claim 17, wherein directing the controller set comprises directing the controller set to add a new middlebox service DCN to the middlebox service DCN set when analyzed message traffic load data indicates that one or more middlebox service DCNs in the middlebox service DCN set are being overutilized.

20. The method of claim 19, wherein

the DCNs comprise virtual machines (VMs);

the controller is a virtualization manager for managing a plurality of VMs that execute on a set of host computers;

the virtualization manager adds a new middlebox service DCN by instantiating a new VM and adding the new VM to the middlebox service DCN set.

21. The method of claim 19, wherein

the middlebox service DCNs comprise virtual machines (VMs);

the controller is a virtualization manager for managing a plurality of VMs that execute on a set of host computers;

the virtualization manager adds a new middlebox service DCN by allocating a previously instantiated VM to the middlebox service DCN set.

22. The method of claim 17 wherein the load balancers are deployed in the egress datapaths of the SCNs and distribute the data messages from the SCNs to the middlebox service DCNs in the middlebox service DCN set.

23. The method of claim 22, wherein at least one load balancer for at least one group of SCNs (i) executes on one host computer with the group of SCNs, and (ii) intercepts data messages sent from the SCNs in the SCN group and directs in a load balanced manner the intercepted data messages to the middlebox service DCNs of the middlebox service DCN set when data messages need to be processed by the middlebox service DCN set.

24. The method of claim 22,

wherein a plurality of SCNs and a plurality of load balancers execute on a set of host computers;

wherein each load balancer in the plurality of load balancers is associated with one SCN, executes on the same host computer at its associated SCN, intercepts data messages sent from its associated SCN and directs in a load balanced manner the intercepted data messages to the middlebox service DCNs of the middlebox service DCN set when data messages need to be processed by the middlebox service DCN set.

25. The method of claim 17, wherein the set of instructions for directing the set of one or more controllers to modify the middlebox service DCN set'"'"'s membership comprises automatically directing the controller to modify the middlebox service DCN set'"'"'s membership without receiving any user input.