CONTROLLER SIDE METHOD OF GENERATING AND UPDATING A CONTROLLER ASSIGNMENT LIST

• US 20150103645A1
• Filed: 11/01/2013
• Published: 04/16/2015
• Est. Priority Date: 10/10/2013
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Shen, Jianjun, Wang, Hua, Stabile, James Joseph, Zhang, Xuan
• Granted Patent: US 9,596,126 B2
• Time in Patent Office: Days
• US Class Current: 370/221
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0659 by isolating or reconfiguring faulty entities
• CPC Class Codes: H04L 41/344 Out-of-band transfers
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 61/103 across network layers, e.g. resolution of network layer into physical layer addresses or address resolution protocol [ARP]
• CPC Class Codes: H04L 61/5014 using dynamic host configuration protocol [DHCP] or bootstrap protocol [BOOTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Some embodiments provide a novel method for distributing control-channel communication load between multiple controllers in a network control system. In some embodiments, the controllers manage physical forwarding elements that forward data between several computing devices (also called hosts or host computers), some or all of which execute one or more virtual machines (VMs). The method of some embodiments distributes a controller assignment list to the host computers. The host computers use this list to identify the controllers with which they need to interact to perform some of the forwarding operations of their associated logical forwarding elements. In some embodiments, agents executing on the host computers (1) review the controller assignment list to identify the appropriate controllers, and (2) establish control channel communications with these controllers to obtain the needed data for effectuating the forwarding operations of their associated physical forwarding elements. These agents in some embodiments are responsible for out-of-band control channel communication with the controllers.

Claims

1. For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:

selecting a common range of index values for different types of LFEs;

identifying a plurality of controllers to manage the LFEs;

generating a controller assignment list that associates each controller with at least one index value in the common index value range, wherein the index values are derivable from identifiers of the LFEs;

distributing the controller assignment list so that the list is received by the hosts, said list for the hosts to use to identify which controllers to request data necessary for effectuating a subset of the operations of the LFEs.

2. The method of claim 1, wherein the controller assignment list associates each controller with a range of index value ranges.

3. The method of claim 1, wherein when first and second index values that are derivable from first and second identifiers of first and second LFEs fall within a particular range of index values, a controller that is associated with the particular index value range manages both the first and second LFEs.

4. The method of claim 3, wherein the first and second LFEs have the same type.

5. The method of claim 3, wherein the first and second LFEs are different type LFEs.

6. The method of claim 5, wherein the first LFE is a logical switch, while the second LFE is a logical router.

7. The method of claim 1, wherein the controller assignment list associates each controller with more than one index value.

8. The method of claim 1, wherein when a particular index value is derivable from first and second identifiers of first and second LFEs of the same type, a controller that is associated with the particular index value manages both the first and second LFEs.

9. The method of claim 1, wherein when a particular index value is derivable from first and second identifiers of first and second LFEs of different types, a controller that is associated with the particular index value manages the first and second LFEs.

10. The method of claim 1, wherein the index values are derivable from the LFE identifiers by using a hash operation that comprises a modulo operation, wherein the common range of index values is associated with a range of remainder values of the modulo operation.

11. For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:

identifying a plurality of controllers to manage the LFEs;

generating a controller assignment list that associates each controller with an index value that is derivable from identifiers of the LFEs, said generating comprising assigning the index values to the controllers by using a load-balancing process for distributing the LFE management load across the controllers;

distributing the controller assignment list so that the list is received by the hosts, said hosts using the list to identify the controllers to contact to obtain data necessary for effectuating a subset of the LFE operations.

12. The method of claim 11, wherein the load-balancing process distributes the responsibility for the LFEs based on a set of criteria that includes assigning substantially the same number of LFEs to each controller.

13. The method of claim 11, wherein the load-balancing process distributes the responsibility for the LFEs based on a set of criteria that includes assigning substantially the same number of LFEs of the same type to each controller.

14. The method of claim 11 further comprising:

detecting that one of the controllers has failed;

distributing the LFEs assigned to the failed controller to the remaining controllers by distributing the index values assigned to the failed controller to the remaining controllers.

15. The method of claim 14, wherein distributing the index values comprises using the load-balancing process to distribute the index values that were previously assigned to the failed controller to the remaining controllers.

16. The method of claim 11 further comprising:

detecting that a new controller has been added;

assigning a subset of LFEs that were previously assigned to other controllers to the new controller by re-assigning a subset of index values that were previously assigned to the other controllers to the new controller.

17. The method of claim 16, wherein distributing the index values comprises using the load-balancing process to distribute an equal-sized or a substantially equal sized portion of index values that were previously assigned to each of the other controllers to the new controller.

18. The method of claim 11, wherein distributing the controller assignment list comprises directly sending the controller assignment list to the host computers.

19. The method of claim 11, wherein distributing the controller assignment list comprises sending the controller assignment list to the other controllers, wherein each controller provides the controller assignment list to a different set of host computers.

20. The method of claim 11, wherein the identifying, generating and distributing operations are performed by one controller that is designated as a master controller for generating the controller assignment list.

21. The method of claim 11 further comprising:

generating an index value based on the identifier of each LFE; and

using the generated index value to specify a controller on the controller assignment list.

22. The method of claim 11 further comprising:

specifying a range of index values that corresponds to a range of remainder values for a modulo operation of a hash operation, wherein the hash operation is used by the host computers to generate the index values from the LFE identifiers.

23. For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:

identifying a plurality of controllers to manage the LFEs;

gathering statistics regarding operations of the host computers relating to the LFEs;

based on the gathered statistics, assigning at least a subset of the LFEs to a subset of controllers;

receiving a request from a particular host computer to connect to a controller to receive data for processing packets associated with a particular LFE;

supplying the identity of a particular controller to the particular host computer, wherein the particular host computer uses the supplied identity to establish a control-channel connection with the particular controller, and uses the connection to process a subset of the operations associated with the particular LFE.

24. The method of claim 23, wherein the particular host computer stores information about the connection in a connection list and then uses the connection list to subsequently identify the particular controller when the host computer needs a controller to process a packet associated with the particular LFE.