Hierarchical controller clusters for interconnecting two or more logical datapath sets

• US 10,091,028 B2
• Filed: 08/17/2012
• Issued: 10/02/2018
• Est. Priority Date: 08/17/2011
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Casado, Martin, Thakkar, Pankaj, Zhang, Ronghua, Wendlandt, Daniel J.
• Primary Examiner(s): Nawaz, Asad
• Assistant Examiner(s): Harley, Jason
• Application Number:
  US13/589,035
• Publication Number:
  US 20130044761A1
• Time in Patent Office: 2,237 Days
• Field of Search: 370401, 370369
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 41/044 comprising hierarchical management structures
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 45/04 Interdomain routing, e.g. hierarchical routing

Abstract

Some embodiments provide a novel network control system for managing a set of switching elements in a network. The network control system includes a first set of network controllers for managing a first set of switching elements that enable communication between a first set of machines. The network control system includes a second set of network controllers for managing a second set of switching elements that enable communication between a second set of machines. The second set of switching elements is separate from the first set of switching elements and the second set of machines is separate from the first set of machines. The network control system includes a third set of network controllers for managing the first and second sets of network controllers in order to enable communication between machines in the first set of machines and machines in the second set of machines.

Claims

1. A network control system for managing a set of switching elements in a network, the network control system comprising:

a first set of network controllers for managing a first set of forwarding elements, that enable communication between a first set of machines, by defining a first logical datapath set to which the machines of the first set logically connect;

a second set of network controllers for managing a second set of forwarding elements, that enable communication between a second set of machines, by defining a second logical datapath set to which the machines of the second set logically connect, the second set of forwarding elements separate from the first set of forwarding elements and the second set of machines separate from the first set of machines; and

a third set of network controllers for managing the first and second sets of network controllers, in order to enable communication between machines in the first set of machines and machines in the second set of machines, by defining a third logical datapath set to which the machines of the first and second sets of machines logically connect.

2. The network control system of claim 1, wherein the first set of forwarding elements and second set of forwarding elements comprise software forwarding elements operating on host devices.

3. The network control system of claim 2, wherein the machines comprise virtual machines, wherein a particular virtual machine operates on a same host device as a particular software forwarding element.

4. The network control system of claim 1, wherein the first set of forwarding elements comprises a first interconnection forwarding element for connecting to an external network and the second set of forwarding elements comprises a second interconnection switching element for connecting to the external network.

5. The network control system of claim 4, wherein the first set of forwarding elements further comprises a first set of edge forwarding elements that each directly couple to at least one machine in the first set of machines, wherein the second set of forwarding elements further comprises a second set of edge forwarding elements that each directly couple to at least one machine in the second set of machines.

6. The network control system of claim 5, wherein the first set of edge forwarding elements are for forwarding network traffic packets, sent from the first set of machines to the second set of machines, to the first interconnection forwarding element in order for the first interconnection forwarding element to forward the network traffic packets to the second interconnection forwarding element and the second interconnection forwarding element to forward the network traffic packets to the second set of edge forwarding elements.

7. The network control system of claim 1, wherein each of the sets of network controllers comprises a single controller instance.

8. The network control system of claim 1, wherein each of the sets of network controllers comprises a plurality of controller instances acting as a distributed controller cluster.

9. The network control system of claim 1, wherein the first set of network controllers, first set of forwarding elements, and first set of machines are located in a first data center while the second set of network controllers, second set of forwarding elements, and second set of machines are located in a second data center.

10. The network control system of claim 9, wherein the third set of network controllers is located in the first data center.

11. The network control system of claim 10, wherein the first data center has a more reliable connection to an interconnecting network than the second data center.

12. The network control system of claim 9, wherein when the second data center is cut off from the first data center, the second set of managed forwarding elements continues to enable communications between the second set of machines.

13. The network control system of claim 9, wherein the third set of network controllers is located in a location separate from the first and second data centers that has a reliable connection to both of the data centers.

14. The network control system of claim 1, wherein the first, second, and third sets of network controllers are located in a single data center.

15. The network control system of claim 1 further comprising a fourth set of network controllers for managing a third set of forwarding elements, that enable communication between a third set of machines, by defining a fourth logical datapath set to which the machines of the third set logically connect, the third set of forwarding elements separate from the first and second sets of forwarding elements and the third set of machines separate from the first and second sets of machines.

16. The network control system of claim 15, wherein the third set of network controllers is further for managing the fourth set of network controllers in order to enable communication between machines in the first, second, and third sets of machines, wherein the machines of the third set of machines logically connect to the third logical datapath set.

17. A network system comprising:

a first set of machines in a first domain and a second set of machines in a second domain;

within each of the domains, a plurality of edge forwarding elements that are each for (i) coupling to the machines in the domain and (ii) forwarding network data to and from the machines in the domain;

a first set of network controllers in the first domain and a second set of network controllers in the second domain, each of the first and second sets of network controllers for managing the edge forwarding elements within their particular domain by specifying a first level logical forwarding element comprising logical ports to which the machines of the set of machines within the particular domain couple, the first level logical forwarding element for implementation by the edge forwarding elements within the domain; and

a third set of network controllers for (i) specifying a second level logical forwarding element comprising logical ports to which machines of both the first and second sets of machines couple and (ii) communicating with the first and second sets of network controllers in order for the first and second sets of network controllers to implement the second level logical forwarding element within the specification of the respective first level logical forwarding elements.

18. The network system of claim 17 further comprising a first interconnection forwarding element in the first domain for forwarding network data between forwarding elements in the first domain and a second interconnection forwarding element in the second domain, the second interconnection forwarding element for forwarding network data between forwarding elements in the second domain and the first interconnection forwarding element.

19. The network system of claim 18, wherein the first and second interconnection forwarding elements are extenders.

20. The network system of claim 18, wherein the first interconnection forwarding element is a cluster of extenders.