Controller side method of generating and updating a controller assignment list

• US 9,596,126 B2
• Filed: 11/01/2013
• Issued: 03/14/2017
• Est. Priority Date: 10/10/2013
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Shen, Jianjun, Wang, Hua, Stabile, James Joseph, Zhang, Xuan
• Primary Examiner(s): HAILE, AWET A
• Application Number:
  US14/070,448
• Publication Number:
  US 20150103645A1
• Time in Patent Office: 1,229 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0659 by isolating or reconfiguring faulty entities
• CPC Class Codes: H04L 41/344 Out-of-band transfers
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 61/103 across network layers, e.g. resolution of network layer into physical layer addresses or address resolution protocol [ARP]
• CPC Class Codes: H04L 61/5014 using dynamic host configuration protocol [DHCP] or bootstrap protocol [BOOTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Some embodiments provide a novel method for distributing control-channel communication load between multiple controllers in a network control system. In some embodiments, the controllers manage physical forwarding elements that forward data between several computing devices (also called hosts or host computers), some or all of which execute one or more virtual machines (VMs). The method of some embodiments distributes a controller assignment list to the host computers. The host computers use this list to identify the controllers with which they need to interact to perform some of the forwarding operations of their associated logical forwarding elements. In some embodiments, agents executing on the host computers (1) review the controller assignment list to identify the appropriate controllers, and (2) establish control channel communications with these controllers to obtain the needed data for effectuating the forwarding operations of their associated physical forwarding elements. These agents in some embodiments are responsible for out-of-band control channel communication with the controllers.

Claims

1. For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:

defining a range of index values for different types of LFEs;

identifying a plurality of controllers to manage the LFEs;

generating a controller assignment list that associates each controller with at least one index value sub-range that contains at least one index value derivable from the identifier of at least one LFE;

distributing the controller assignment list to host computers, wherein the host computers use the list to identify which controllers to request data necessary for effectuating a subset of the operations of the LFEs, wherein when first and second index values derived from first and second identifiers of first and second LFEs fall within a particular sub-range of index values, a controller associated with the particular index value sub-range manages both the first and second LFEs.

2. The method of claim 1, wherein the controller assignment list associates different controllers with different sub-ranges of index values within the defined index value range.

3. The method of claim 1, wherein the first and second LFEs have the same type.

4. The method of claim 1, wherein the first and second LFEs are different type LFEs.

5. The method of claim 4, wherein the first LFE is a logical switch, while the second LFE is a logical router.

6. The method of claim 1, wherein the controller assignment list associates each controller with more than one index value.

7. The method of claim 1, wherein when a particular index value is derivable from first and second identifiers of first and second LFEs of the same type, a controller that is associated with the particular index value manages both the first and second LFEs.

8. The method of claim 1, wherein when a particular index value is derivable from first and second identifiers of first and second LFEs of different types, a controller that is associated with the particular index value manages the first and second LFEs.

9. The method of claim 1, wherein the index values are derivable from the LFE identifiers by using a hash operation that comprises a modulo operation.

10. For a network control system that includes a plurality of host computers that execute a plurality of physical forwarding elements (PFEs) that collectively implement a plurality of logical forwarding elements (LFEs), a method of managing the LFEs comprising:

identifying a plurality of controllers to manage the LFEs;

generating a controller assignment list that associates different controllers with different index value ranges, at least one index value range containing at least one index value that is derivable from the identifier of at least one LFE, said generating comprising assigning the index values to the controllers by using a load-balancing process for distributing the LFE management load across the controllers;

distributing the controller assignment list to the host computers, wherein the host computers use the list to identify which controllers to contact to request data necessary for effectuating a subset of the LFE operations, wherein when first and second index values derived from first and second identifiers of first and second LFEs fall within a particular range of index values, a controller associated with the particular index value range manages both the first and second LFEs.

11. The method of claim 10, wherein the load-balancing process distributes the responsibility for the LFEs based on a set of criteria that includes assigning the same number of LFEs to each controller.

12. The method of claim 10, wherein the load-balancing process distributes the responsibility for the LFEs based on a set of criteria that includes assigning the same number of LFEs of the same type to each controller.

13. The method of claim 10 further comprising:

detecting that one of the controllers has failed; and

distributing the LFEs assigned to the failed controller to the remaining controllers by distributing the index value range assigned to the failed controller to the remaining controllers.

14. The method of claim 13, wherein distributing the index values comprises using the load-balancing process to distribute the index value range that was previously assigned to the failed controller to the remaining controllers.

15. The method of claim 10 further comprising:

detecting that a new controller has been added; and

assigning a subset of LFEs that were previously assigned to other controllers to the new controller by re-assigning a subset of index values that were previously assigned to the other controllers to the new controller.

16. The method of claim 15, wherein distributing the index values comprises using the load-balancing process to distribute an equal-sized portion of index values that were previously assigned to each of the other controllers to the new controller.

17. The method of claim 10, wherein distributing the controller assignment list comprises directly sending the controller assignment list to the host computers.

18. The method of claim 10, wherein distributing the controller assignment list comprises sending the controller assignment list to the other controllers, wherein each controller provides the controller assignment list to a different set of host computers.

19. The method of claim 10, wherein the identifying, generating and distributing operations are performed by one controller that is designated as a master controller for generating the controller assignment list.

20. The method of claim 10,

wherein the range of index values corresponds to a range of remainder values for a modulo operation of a hash operation, wherein the hash operation is used by the host computers to generate the index values from the LFE identifiers.