Federating interconnection switching element network to two or more levels

• US 9,137,052 B2
• Filed: 08/17/2012
• Issued: 09/15/2015
• Est. Priority Date: 08/17/2011
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Koponen, Teemu, Casado, Martin, Thakkar, Pankaj, Zhang, Ronghua
• Primary Examiner(s): Hsu, Alpus H
• Assistant Examiner(s): WILLIAMS, ELTON S
• Application Number:
  US13/589,049
• Publication Number:
  US 20130044641A1
• Time in Patent Office: 1,124 Days
• Field of Search: 370/255, 370/401, 370/389, 370/392, 370/351, 370/230, 370/235, 370/408, 709/238, 709/252
• US Class Current: 1/1
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 41/044 comprising hierarchical management structures
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 45/04 Interdomain routing, e.g. hierarchical routing

Abstract

A network control system for interconnecting several separate networks. The system includes i) several interconnection switching elements, each of which is for connecting one of the separate networks to a common interconnecting network, ii) a first set of network controllers for managing a first set of the interconnection switching elements at a first set of separate networks in order for machines at different networks within the first set to communicate with each other, iii) a second set of network controllers for managing a second set of interconnection switching elements at a second set of separate networks in order for machines at different networks within the second set to communicate with each other, and iv) a third set of network controllers for managing the first and second sets of network controllers in order for machines at networks in the first set to communicate with machines at networks in the second set.

Claims

1. A network control system for interconnecting a plurality of separate networks arranged in a plurality of domains at at least two different levels, wherein a first domain at a first level contains at least two domains at a second level, the system comprising:

a plurality of interconnection switching elements, each interconnection switching element in the plurality for connecting one of the separate networks to a common interconnecting network; and

a first set of network controllers for managing a first set of the interconnection switching elements at a first set of the separate networks that form a first domain at the second level by defining a first logical datapath set, the first logical datapath set having logical ports to which networks within the first set of separate networks couple and to which at least a second logical datapath set couples, the first logical datapath set for implementation by the first set of interconnection switching elements in order for machines at the first set of separate networks to communicate with each other, wherein at least one of the network controllers in the first set of network controllers is for managing a plurality of the interconnection switching elements of the first set of interconnection switching elements;

a second set of network controllers for managing a second set of the interconnection switching elements at a second set of the separate networks that form a second domain at the second level by defining the second logical datapath set, the second logical datapath set having logical ports to which networks within the second set of separate networks couple and to which at least the first logical datapath set couples, the second logical datapath set for implementation by the second set of interconnection switching elements in order for machines at the second set of separate networks to communicate with each other, wherein at least one of the network controllers in the second set of network controllers is for managing a plurality of the interconnection switching elements of the second set of interconnection switching elements; and

a third set of network controllers for managing the first and second sets of network controllers by defining a third logical datapath set having logical ports to which the networks in the first and second sets of separate networks couple, the third logical datapath set for implementation by the first and second sets of interconnection switching elements in order for machines at the first set of separate networks to communicate with machines at the second set of separate networks, wherein the first and second sets of separate networks belong to the first domain at the first level, which contains the first and second domains at the second level,

wherein each network controller in the first, second, and third sets of network controllers is for execution by at least one processing unit of a server.

2. The network control system of claim 1, wherein each of the separate networks is a segmented network.

3. The network control system of claim 2, wherein each of the segmented networks is a segmented layer 2 (“

L2”

) network and the common interconnecting network is a layer 3 (“

L3”

) network.

4. The network control system of claim 1, wherein the plurality of interconnection switching elements are extenders.

5. The network control system of claim 1 further comprising a fourth set of network controllers for managing a third set of the interconnection switching elements at a third set of the separate networks that form a third domain at the second level by defining a fourth logical datapath set, the fourth logical datapath set having logical ports to which networks within the third set of separate networks couple and to which at least the first and second logical datapath sets couple, the fourth logical datapath set for implementation by the third set of interconnection switching elements in order for machines at the third set of separate networks to communicate with each other, wherein at least one of the network controllers in the fourth set of network controllers is for managing a plurality of the interconnection switching elements of the third set of interconnection switching elements.

6. The network control system of claim 5, wherein the third set of network controllers is further for managing the fourth set of network controllers in order for machines at networks in the third set of separate networks to communicate with machines at networks in the first and second sets of separate networks, wherein the third logical datapath set has logical ports to which the networks in the third set of separate networks couple, the third logical datapath set further for implementation by the interconnection switching elements of the third set of interconnection switching elements, wherein the third set of separate networks belongs to the first domain at the first level.

7. The network control system of claim 6 further comprising a fifth set of network controllers for managing a fourth set of interconnection switching elements at a fourth set of the separate networks that form a fourth domain at the second level by defining a fifth logical datapath set, the fifth logical datapath set having logical ports to which networks within the fourth set of separate networks couple and to which at least the first, second, and fourth logical datapath sets couple, the fifth logical datapath set for implementation by the fourth set of interconnection switching elements in order for machines at the fourth set of separate networks to communicate with each other, wherein at least one of the network controllers in the fifth set of network controllers is for managing a plurality of the interconnection switching elements of the fourth set of interconnection switching elements, wherein the third set of network controllers is further for managing the fifth set of network controllers in order for machines at the fourth set of separate networks to communicate with machines at networks in the first, second, and third sets of separate networks, wherein the third logical datapath set has logical ports to which the networks in the fourth set of separate networks couple, the third logical datapath set further for implementation by the interconnection switching elements of the fourth set of interconnection switching elements, wherein the fourth set of separate networks belongs to the first domain at the first level.

8. The network control system of claim 7, wherein the third set of controllers is for determining a path for a packet to travel from the first set of interconnection switching elements to the fourth set of interconnection switching elements.

9. The network control system of claim 1 further comprising:

a fourth set of network controllers for managing a third set of the interconnection switching elements at a third set of the separate networks that form a third domain at the second level by defining a fourth logical datapath set, the fourth logical datapath set having logical ports to which networks within the third set of separate networks couple and to which at least a fifth logical datapath set couples, the fourth logical datapath set for implementation by the third set of interconnection switching elements in order for machines at the third set of separate networks to communicate with each other, wherein at least one of the network controllers in the fourth set of network controllers is for managing a plurality of the interconnection switching elements of the third set of interconnection switching elements;

a fifth set of network controllers for managing a fourth set of the interconnection switching elements at a fourth set of the separate networks that form a fourth domain at the second level by defining the fifth logical datapath set, the fifth logical datapath set having logical ports to which networks within the fourth set of separate networks couple and to which at least the fourth logical datapath set couples, the fifth logical datapath set for implementation by the fourth set of interconnection switching elements in order for machines at the fourth set of separate networks to communicate with each other, wherein at least one of the network controllers in the fifth set of network controllers is for managing a plurality of the interconnection switching elements of the fourth set of interconnection switching elements; and

a sixth set of network controllers for managing the fourth and fifth sets of network controllers by defining a sixth logical datapath set having logical ports to which the networks in the third and fourth sets of separate networks couple, the sixth logical datapath set for implementation by the third and fourth sets of interconnection switching elements in order for machines at the third set of separate networks to communicate with machines at the fourth set of separate networks, wherein the third and fourth sets of separate networks belong to a second domain at the first level that contains the third and fourth domains at the second level.

10. The network control system of claim 9 further comprising:

a seventh set of network controllers for managing the third and sixth sets of network controllers by defining a seventh logical datapath set having logical ports to which the networks in the first, second, third, and fourth sets of separate networks couple, the seventh logical datapath set for implementation by the first, second, third, and fourth sets of interconnection switching elements in order for machines at the first and second sets of separate networks to communicate with machines at the third and fourth sets of separate networks, wherein the first, second, third, and fourth sets of separate networks belong to a domain at a third level that contains the first and second domains at the first level.

11. The network control system of claim 1, wherein the first, second, and third sets of network controllers each comprise a single controller instance.

12. The network control system of claim 1, wherein the first, second, and third sets of network controllers each comprise a plurality of controller instances acting as a distributed controller cluster.

13. A method for forwarding a packet at an interconnection switching element in a managed network of interconnection switching elements, each interconnection switching element located at one of a plurality of segmented networks, the segmented networks arranged in a plurality of domains at a plurality of different levels, wherein a plurality of the domains at a first level contains more than one domain at a next lower level, wherein the interconnection switching elements located in a particular domain at a particular level implement a logical datapath set that couples to network segments located within the particular domain as well as to other domains of the particular level within the domain at a next highest level that contains the particular domain, the method comprising:

at a first managed interconnection switching element located at a first network in a first domain of the lowest level, receiving a packet from a machine on a first network segment in the first network, the packet having an address identifying a destination machine located on a second network segment in a second, different network, the second network located in a second domain of the lowest level, the second domain different than the first domain;

removing a context tag that identifies the network segment within the first network;

based on a mapping of the destination address of the packet to a particular destination network, encapsulating the packet with a first context identifier that identifies a first logical egress port of a first logical datapath set at a highest level that couples to all of the networks in all of the lowest-level domains, the first logical datapath set implemented by the interconnection switching elements at all of the lowest-level domains;

based on a mapping of the first logical port to subsequent logical ports of logical datapath sets at each of the next lowest levels, encapsulating the packet with an additional logical context identifier for each level, each of the additional logical context identifiers identifying a logical egress port of a logical datapath set at a lower level mapped from a port of a logical datapath set at the next higher level, wherein each of the logical datapath sets has a particular level and defines mappings between (i) logical ports of a logical datapath set at the next level higher and its logical ports and (ii) logical ports of one or more logical datapath sets at the next level lower and its logical ports; and

transmitting the packet with a plurality of encapsulations out of a port of the interconnection switching element based on a lowest-level context identifier.

14. The method of claim 13, wherein the first network segment is segmented using virtual local area network (VLAN) tagging and the second network segment is segmented using multi-protocol labeling system (MPLS) labeling.

15. The method of claim 13, wherein the first network segment is segmented using a first VLAN technique and the second network segment is segmented using a second VLAN technique.

16. The method of claim 13, wherein removing a context tag that identifies the network segment comprises using stored instructions that indicate particular bits of the packet that contain the context tag.

17. The method of claim 13 further comprising, after removing the context tag and before encapsulating the packet:

mapping the identified network segment to a particular logical ingress port of a lowest-level logical datapath set; and

for each particular level of logical datapath set above the lowest level, mapping an identified logical ingress port of the next lowest-level logical datapath set to a logical ingress port of the particular level logical datapath set.

18. The method of claim 17 further comprising storing each of the logical ingress ports in registers of the interconnection switching element until the packet is transmitted out of the port of the interconnection switching element.

19. The method of claim 13, wherein the destination address is one of a plurality of destination addresses that map to the particular destination network, wherein the particular destination network couples to the first logical egress port.

20. The method of claim 13, wherein each of the segmented networks has at least one of the interconnection switching elements located at the segmented network.