Distributed network control system with one master controller per managed switching element

• US 9,106,587 B2
• Filed: 08/25/2011
• Issued: 08/11/2015
• Est. Priority Date: 07/06/2010
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Casado, Martin, Stribling, Jeremy, Gude, Natasha, Lambeth, W. Andrew
• Primary Examiner(s): Patel, Chirag R
• Application Number:
  US13/218,427
• Publication Number:
  US 20130058348A1
• Time in Patent Office: 1,447 Days
• Field of Search: 709208-211
• US Class Current: 1/1
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 15/17312 Routing techniques specific to parallel machines, e.g. wormhole, store and forward, shortest path problem congestion routing on a LAN H04L45/00
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 2101/622 Layer-2 addresses, e.g. medium access control [MAC] addresses
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0853 by actively collecting configuration information or by backing up configuration information
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 45/02 Topology update or discovery
• CPC Class Codes: H04L 45/036 Updating the topology between route computation elements, e.g. between OpenFlow controllers
• CPC Class Codes: H04L 45/586 of virtual routers
• CPC Class Codes: H04L 45/76 Routing in software-defined topologies, e.g. routing between virtual machines
• CPC Class Codes: H04L 47/783 Distributed allocation of resources, e.g. bandwidth brokers
• CPC Class Codes: H04L 49/00 Packet switching elements
• CPC Class Codes: H04L 49/1546 using pipelined operation
• CPC Class Codes: H04L 49/3063 Pipelined operation
• CPC Class Codes: H04L 49/70 : Virtual switches
• CPC Class Codes: H04L 61/5007 : Internet protocol [IP] addresses

Abstract

A network control system for managing several switching elements. The network control system includes first and second controllers for generating data for managing first and second sets of switching elements. The first controller is further for serving as a master controller of the first set of switching elements. The second controller is further for serving as a master controller of the second set of switching elements. The master controller for a particular set of switching elements is the only controller that is allowed to propagate data to the particular set of switching elements data for managing the particular set of switching elements.

Claims

1. A network control system for managing a plurality of switching elements, each switching element for forwarding data packets in the network, the network control system comprising:

first and second controllers for receiving logical datapath sets and for generating data that, once propagated to the switching elements, enables the switching elements to implement the logical datapath sets by managing forwarding behaviors of the switching elements,

the first controller further for serving as a master controller for a first logical datapath set and for a first set of switching elements,

the second controller further for serving as a master controller for a second logical datapath set and for a second set of switching elements, wherein the first controller is further for sending data to the second controller that the first controller generates for managing the second set of switching elements to implement the first logical datapath set,

wherein the master controller for a particular set of switching elements is the only controller that is allowed to propagate data to the particular set of switching elements for managing the forwarding behaviors of the particular set of switching elements, wherein the master controller for a particular logical datapath set is the only controller that generates data for propagating to a group of switching elements that implement the particular logical datapath set.

2. The network control system of claim 1, wherein the first and second controllers receive the first and second logical datapath sets in terms of logical control plane data.

3. The network control system of claim 2, wherein the first logical datapath set is implemented by switching elements in both the first and second sets of switching elements.

4. The network control system of claim 1, wherein the second controller is further for generating data for managing the forwarding behaviors of at least one switching element in the first set of switching elements.

5. The network control system of claim 4, wherein the data generated by the second controller for the first-set switching element is propagated to the first controller to allow the first controller to propagate the generated data to the first-set switching element.

6. The network control system of claim 5,

wherein each controller has a network information base (NIB) storage that stores data regarding one or more switching elements,

wherein the data generated by the second controller for the first-set switching element is propagated to the first controller by transferring the data to the first controller'"'"'s NIB storage.

7. The network control system of claim 6, wherein the data generated by the second controller for the first-set switching element is transferred to the first controller'"'"'s NIB storage by first storing the data in the second controller'"'"'s NIB storage and then replicating said data in the first controller'"'"'s NIB storage.

8. The network control system of claim 7,

wherein each controller has a secondary storage for storing at least a portion of the data stored in the NIB storage of the controller;

wherein the generated data for the first-set switching element is transferred from the second controller'"'"'s NIB storage to the NIB storage of the first controller through replication of the secondary storages of the first and second controllers.

9. The network control system of claim 1, wherein the data generated by each controller comprises physical control plane data that the controller generates from logical control plane data, the logical control plane data specifying logical datapath sets that are implemented by the switching elements based on the generated physical control plane data.

10. The network control system of claim 9, wherein each controller generates the physical control plane data by converting the logical control plane data to logical forwarding plane data and then converting the logical forwarding plane data to the physical control plane data.

11. The network control system of claim 1, wherein the first controller is not allowed to propagate data to the second set of switching elements while the second controller serves as the master controller for the second set of switching elements.

12. The network control system of claim 1, wherein upon failure of the second controller, the first controller is elected as the master of the second set of switching elements, and as the master, is solely responsible for propagating data to the second set of switching elements.

13. A method for managing a plurality of switching elements, the method comprising:

providing a first controller for serving as a master controller for a first logical datapath set and for a first set of switching elements, wherein as the master controller of the first logical datapath set, the first controller generates data for managing the first set of switching elements and a second set of switching elements to implement the first logical datapath set; and

providing a second controller for serving as a master controller for a second logical datapath set and for the second set of switching elements, wherein as the master of the second logical datapath set, the second controller generates data for managing the second set of switching elements and a third set of switching elements to implement the second logical datapath set,

wherein the master controller for a particular set of switching elements is the only controller that is allowed to propagate data to the particular set of switching elements, wherein the first controller is further for sending data to the second controller that the first controller generates for managing the second set of switching elements to implement the first logical datapath set.

14. The method of claim 13, wherein the first and second controllers are further for receiving logical datapath sets.

15. The method of claim 14,

wherein the master controller for a particular logical datapath set is the only controller that generates data for propagating to a group of switching elements that implement the particular logical datapath set.

16. The method of claim 13, wherein at least one switching element in the third set of switching elements is also in the first set of switching elements.

17. The method of claim 16, wherein the data generated by the second controller for the first-set switching element is propagated to the first controller to allow the first controller to propagate the generated data to the first-set switching element.

18. The method of claim 17,

wherein each controller has a network information base (NIB) storage that stores data regarding one or more switching elements,

wherein the data generated by the second controller for the first-set switching element is propagated to the first controller by transferring the data to the first controller'"'"'s NIB storage.

19. The method of claim 18, wherein the data generated by the second controller for the first-set switching element is transferred to the first controller'"'"'s NIB storage by first storing the data in the second controller'"'"'s NIB storage and then replicating said data in the first controller'"'"'s NIB storage.

20. The method of claim 19,

wherein each controller has a secondary storage for storing at least a portion of the data stored in the NIB storage of the controller;

wherein the generated data for the first-set switching element is transferred from the second controller'"'"'s NIB storage to the NIB storage of the first controller through replication of the secondary storages of the first and second controllers.

21. The method of claim 13, wherein the data generated by each controller comprises physical control plane data that the controller generates from logical control plane data, the logical control plane data specifying the logical datapath set that is implemented by the switching elements based on the generated physical control plane data.

22. The method of claim 21, wherein each controller generates the physical control plane data by converting the logical control plane data to logical forwarding plane data and then converting the logical forwarding plane data to the physical control plane data.

23. The method of claim 13, wherein the first controller is not allowed to propagate data to the second set of switching elements while the second controller serves as the master controller for the second set of switching elements.

24. The method of claim 13, wherein upon failure of the second controller, the first controller is elected as the master controller of the second set of switching elements, and as the master controller, is solely responsible for propagating data to the second set of switching elements.