Distributed control platform for large-scale production networks

• US 8,830,823 B2
• Filed: 07/06/2011
• Issued: 09/09/2014
• Est. Priority Date: 07/06/2010
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Casado, Martin, Gude, Natasha, Stribling, Jeremy
• Primary Examiner(s): Shingles, Kristie
• Application Number:
  US13/177,538
• Publication Number:
  US 20130060929A1
• Time in Patent Office: 1,161 Days
• Field of Search: 709/224, 370/219
• US Class Current: 370/219
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 15/17312 Routing techniques specific to parallel machines, e.g. wormhole, store and forward, shortest path problem congestion routing on a LAN H04L45/00
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 2101/622 Layer-2 addresses, e.g. medium access control [MAC] addresses
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0853 by actively collecting configuration information or by backing up configuration information
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/036 Updating the topology between route computation elements, e.g. between OpenFlow controllers
• CPC Class Codes: H04L 45/586 of virtual routers
• CPC Class Codes: H04L 45/76 Routing in software-defined topologies, e.g. routing between virtual machines
• CPC Class Codes: H04L 47/783 Distributed allocation of resources, e.g. bandwidth brokers
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/1546 using pipelined operation
• CPC Class Codes: H04L 49/3063 Pipelined operation
• CPC Class Codes: H04L 49/70 : Virtual switches
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses

Abstract

Some embodiments provide a program that monitors several network controllers in a network. Each network controller manages a logical data path set that is implemented across several managed switching elements. The program detects that a first network controller in the several network controllers has failed. The program selects a second network controller in the several network controllers to manage the logical data path set managed by the first network controller.

Claims

1. A non-transitory machine readable medium storing a program executable by at least one processing unit, the program comprising sets of instructions for:

monitoring a plurality of network controllers for managing a plurality of managed switching elements, the plurality of network controllers comprising a first network controller for (i) defining a logical data path set (LDPS) in terms of logical forwarding plane (LFP) data that specifies logical forwarding operations of a logical switching element that implements the LDPS and (ii) converting the LFP data to physical control plane (PCP) data for supplying to a set of managed switching elements across which the LDPS is implemented;

detecting that the first network controller in the plurality of network controllers has failed; and

selecting a second network controller in the plurality of network controllers to perform operations of the first network controller for the LDPS.

2. The non-transitory machine readable medium of claim 1, wherein the first network controller defines the LDPS in terms of the LFP data by receiving logical control plane (LCP) data that specifies the LDPS and converting the LCP data to LFP data.

3. The non-transitory machine readable medium of claim 1, wherein a managed switching element that receives the PCP data converts the PCP data into physical forwarding plane data that specifies a forwarding behavior of the managed switching element.

4. The non-transitory machine readable medium of claim 1, wherein the first network controller allows a user to configure the logical switching element without configuring the managed switching elements.

5. The non-transitory machine readable medium of claim 1, wherein the set of instructions for detecting comprises a set of instructions for determining that the first network controller has failed when a response to a polling message from the first network controller is not received within a defined amount of time.

6. For a particular network controller of a plurality of network controllers for managing a plurality of managed switching elements that forward data in a network, a method comprising:

monitoring the plurality of network controllers comprising a first network controller for expressing a logical forwarding plane (LFP) of a logical data path set (LDPS), said LFP comprising a logical L2 forwarding table that specifies logical L2 forwarding operations of a logical switch that is defined by the LDPS and is implemented across a set of managed switching elements;

detecting that the first network controller in the plurality of network controllers has failed; and

selecting a second network controller in the plurality of network controllers to perform operations of the first network controller for the LDPS.

7. The method of claim 6, wherein the first network controller expresses the LFP of the LDPS by receiving logical control plane (LCP) data that specifies the LDPS and converting the LCP data to LFP data.

8. The method of claim 7, wherein the first network controller converts the LFP data to physical control plane (PCP) data for supplying to the plurality of managed switching elements across which the logical switch is implemented.

9. The method of claim 8, wherein the first network controller sends the PCP data to a third network controller in the plurality of network controllers and the third network controller supplies the PCP data to a set of managed switching elements that implements at least a portion of said logical switch.

10. The method of claim 9, wherein the third network controller is the only controller that is configured to send the PCP data to the set of managed switching elements.

11. For a network control system that manages a plurality of managed forwarding elements that forward data within a network, a method for managing the plurality of managed forwarding elements comprising:

grouping the plurality of managed forwarding elements into sets of managed forwarding elements;

instantiating a plurality of network operating system (NOS) instances and configuring each of the NOS instances to maintain forwarding state information of one set of the sets of managed forwarding elements; and

configuring each NOS instance of the plurality of NOS instances to collect the maintained forwarding state information for the one set of managed forwarding elements and exchange the maintained forwarding state information with the rest of the plurality of NOS instances.

12. The method of claim 11 further comprising managing the plurality of managed forwarding elements using an initial NOS instance that maintains forwarding state information of the plurality of managed forwarding elements, wherein the network control system performs said grouping, instantiating, and configuring when an amount of the forwarding state information that the initial NOS instance maintains exceeds a threshold amount for the initial NOS instance to maintain.

13. The method of claim 12, wherein the threshold amount of information that the initial NOS instance is to maintain is exceeded when memory resources that the initial NOS instance uses run out.

14. The method of claim 11 further comprising, when an amount of the forwarding state information obtained by each NOS instance of the plurality of NOS instances from other NOS instances of the plurality of NOS instances exceeds a threshold amount for the NOS instance to handle, configuring at least one particular NOS instance of the plurality of NOS instances (i) to reduce the forwarding state information of the set of managed forwarding elements maintained by the particular NOS instance and (ii) to present the reduced forwarding state information to another NOS instance of the plurality of NOS instances.

15. The method of claim 14,

wherein each managed forwarding element in the plurality of managed forwarding elements has a link established to at least one of the other managed forwarding elements in the plurality of managed forwarding elements,

wherein the forwarding state information of the plurality of managed forwarding elements includes link utilization information for each link of the links,

wherein the particular NOS instance reduces the forwarding state information by aggregating link utilization information for the links of the managed forwarding elements of which the particular NOS instance maintains the forwarding state information.

16. The method of claim 14, wherein the threshold amount of information that each of the plurality of NOS instances is to handle is exceeded when central processing unit (CPU) resources that the NOS instance uses reach a full capacity to process the information.

17. The method of claim 11, wherein each managed forwarding element in the plurality of managed forwarding elements has a link established to each of the other managed forwarding elements in the plurality of managed forwarding elements.

18. The method of claim 11, wherein each NOS instance in the plurality of NOS instances is communicatively connected to each of the other NOS instances in the plurality of NOS instances in order to exchange the forwarding state information.

19. The method of claim 11 further comprising:

configuring a first NOS instance to send a request to a second NOS instance to obtain the forwarding state information that the second NOS instance maintains; and

configuring the second NOS instance to send, upon receiving the request, a response including the requested forwarding state information to the first NOS instance.

20. The method of claim 19, wherein the first NOS instance in the plurality of NOS instances is not communicatively connected to the second NOS instance in the plurality of NOS instances wherein the method further comprises configuring a third NOS instance of the plurality of NOS instances to relay the request to the second NOS instance.

21. The method of claim 20, wherein the third NOS instance is communicatively connected to the first NOS instance and the second NOS instance.