Extension of network control system into public cloud

• US 10,367,757 B2
• Filed: 08/31/2016
• Issued: 07/30/2019
• Est. Priority Date: 08/27/2016
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Chandrashekhar, Ganesan, Hira, Mukesh, Wang, Su, Katrekar, Akshay
• Primary Examiner(s): Neurauter, Jr., Geroge C
• Application Number:
  US15/253,829
• Publication Number:
  US 20180062917A1
• Time in Patent Office: 1,063 Days
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 2209/24 Key scheduling, i.e. generating round keys or sub-keys for block encryption
• CPC Class Codes: H04L 2212/00 Encapsulation of packets
• CPC Class Codes: H04L 41/044 comprising hierarchical management structures
• CPC Class Codes: H04L 41/0806 : for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/0895 : Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/12 : Discovery or management of network topologies
• CPC Class Codes: H04L 41/20 : Network management software packages
• CPC Class Codes: H04L 41/40 : using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 45/38 : Flow based routing
• CPC Class Codes: H04L 45/72 : Routing based on the source address
• CPC Class Codes: H04L 45/74 : Address processing for routing
• CPC Class Codes: H04L 47/125 : by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 47/32 : by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 49/15 : Interconnection of switching modules
• CPC Class Codes: H04L 49/25 : Routing or path finding in a switch fabric
• CPC Class Codes: H04L 49/70 : Virtual switches
• CPC Class Codes: H04L 61/2514 : between local and global IP addresses
• CPC Class Codes: H04L 61/2521 : Translation architectures other than single NAT servers
• CPC Class Codes: H04L 61/2539 : Hiding addresses; Keeping addresses anonymous
• CPC Class Codes: H04L 61/256 : NAT traversal
• CPC Class Codes: H04L 61/2592 : using tunnelling or encapsulation
• CPC Class Codes: H04L 61/5014 : using dynamic host configuration protocol [DHCP] or bootstrap protocol [BOOTP]
• CPC Class Codes: H04L 63/0209 : Architectural arrangements, e.g. perimeter networks or demilitarized zones
• CPC Class Codes: H04L 63/0236 : Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0263 : Rule management
• CPC Class Codes: H04L 63/0272 : Virtual private networks
• CPC Class Codes: H04L 63/029 : Firewall traversal, e.g. tunnelling or, creating pinholes
• CPC Class Codes: H04L 63/0428 : wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/062 : for key distribution, e.g. centrally by trusted party cryptographic mechanisms or cryptographic arrangements for key distribution involving a central third party H04L9/0819
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/0819 : Key transport or distribution, i.e. key establishment techniques where one party creates or otherwise obtains a secret value, and securely transfers it to the other(s) network architectures or network communication protocols for key distribution in a packet data network H04L63/062
• CPC Class Codes: H04L 9/3213 : using tickets or tokens, e.g. Kerberos network architectures or network communication protocols for entities authentication using tickets in a packet data network H04L63/0807

Abstract

Some embodiments provide a method for a first network controller that manages a logical network implemented in a datacenter including forwarding elements to which the first network controller does not have access. The method identifies a first data compute node (DCN) in the datacenter configured to execute a second network controller. The method distributes configuration data defining the logical network to the first DCN. The second network controller distributes sets of the configuration data to local agents executing on additional DCNs in the datacenter that send and receive messages through the logical network. Both managed forwarding elements and the local agents execute on each of the additional DCNs. Each local agent on a particular DCN is for receiving a set of configuration data from the second network controller and configuring the managed forwarding element on the particular DCN to implement the logical network according to the set of configuration data.

Claims

1. For a first network controller that manages a logical network implemented on data compute nodes assigned to a particular tenant in a public multi-tenant datacenter comprising forwarding elements to which the first network controller does not have access, a method comprising:

identifying a first data compute node of the data compute nodes assigned to the particular tenant in the public multi-tenant datacenter that is configured to execute a second network controller; and

distributing configuration data defining the logical network to the first data compute node,

wherein the second network controller executing at the first data compute node distributes sets of the configuration data to a plurality of local agents executing on additional data compute nodes of the data compute nodes assigned to the particular tenant in the public multi-tenant datacenter that send and receive messages through the logical network, wherein both a managed forwarding element and a local agent execute on each of the additional data compute nodes, each local agent on a particular data compute node of the additional data compute nodes for receiving one of the sets of configuration data from the second network controller and configuring the managed forwarding element on the particular data compute node to implement the logical network according to the received set of configuration data.

2. The method of claim 1 further comprising distributing the configuration data to a plurality of local controller agents operating in virtualization software of host machines of a private datacenter operated by the particular tenant of the public multi-tenant datacenter, wherein the first network controller operates in the private datacenter.

3. The method of claim 1, wherein:

the logical network comprises one or more logical switches, each having a plurality of logical ports to which data compute nodes of the logical network connect;

the method further comprises distributing the configuration data to a plurality of local controller agents operating in virtualization software of host machines of a private datacenter; and

the first network controller maps each of the local controller agents in the private datacenter to one or more of the logical ports based on the data compute nodes operating on the host machine of the local controller agent and maps the second network controller to all of the logical ports the data compute nodes for which operate in the public multi-tenant datacenter.

4. The method of claim 1, wherein the second network controller receives the distributed configuration data as a plurality of atomic pieces of data regarding at least one of (i) logical forwarding element and logical port configuration and (ii) distributed firewall rules relating to at least one data compute node operating in the public multi-tenant datacenter, identifies a span for each atomic piece of data, and distributes sets of the atomic pieces of data to the local agents based on the identified spans.

5. The method of claim 1, wherein the additional data compute nodes connect to the forwarding elements of the public multi-tenant datacenter that are not configurable by the first network controller, wherein the forwarding elements of the datacenter execute in virtualization software of host machines on which the additional data compute nodes operate.

6. The method of claim 1, wherein the managed forwarding elements executing on the data compute nodes comprise flow-based forwarding elements, wherein the local agents receive configuration data from the second network controller and convert the received configuration data into flow entries for the forwarding elements.

7. The method of claim 1, wherein the first data compute node and the additional data compute nodes operate in a virtual private cloud of the public multi-tenant datacenter.

8. The method of claim 7, wherein the virtual private cloud is a first virtual private cloud, the method further comprising:

identifying a second data compute node of the data compute nodes assigned to the particular tenant in a second virtual private cloud of the public multi-tenant datacenter, the second data compute node configured to execute a third network controller; and

distributing the configuration data defining the logical network to the second data compute node,

wherein the third network controller executing at the second data compute node distributes sets of the configuration data to a second plurality of local agents executing on a second set of additional data compute nodes of the data compute nodes assigned to the particular tenant in the second virtual private cloud of the public multi-tenant datacenter that send and receive messages through the logical network, wherein both a managed forwarding element and a local agent execute on each of the additional data compute nodes in the second virtual private cloud, each local agent on a particular data compute node of the second set of additional data compute nodes for receiving one of the sets of configuration data from the third network controller and configuring the managed forwarding element on the particular data compute node to implement the logical network according to the received set of configuration data.

9. The method of claim 7, wherein the virtual private cloud is a first virtual private cloud of a first public multi-tenant datacenter, wherein the data compute nodes assigned to the particular tenant further comprise data compute nodes assigned to the particular tenant in a second public multi-tenant datacenter, the method further comprising:

identifying a second data compute node of the data compute nodes assigned to the particular tenant in a second virtual private cloud of the second public multi-tenant datacenter, the second data compute node configured to execute a third network controller; and

distributing the configuration data defining the logical network to the second data compute node,

wherein the third network controller executing at the second data compute node distributes sets of the configuration data to a second plurality of local agents executing on a second set of additional data compute nodes of the data compute nodes assigned to the particular tenant in the second virtual private cloud of the second public multi-tenant datacenter that send and receive messages through the logical network, wherein both a managed forwarding element and a local agent execute on each of the additional data compute nodes in the second virtual private cloud, each local agent on a particular data compute node of the second set of additional data compute nodes for receiving one of the sets of configuration data from the third network controller and configuring the managed forwarding element on the particular data compute node to implement the logical network according to the received set of configuration data.

10. The method of claim 1, wherein the second network controller generates and distributes overlay configuration data to the plurality of local agents, the overlay configuration data for enabling the managed forwarding elements executing on the additional data compute nodes to encapsulate data messages sent between the additional data compute nodes.

11. A non-transitory machine readable medium storing a program which when executed by at least one processing unit implements a first network controller that manages a logical network implemented on data compute nodes assigned to a particular tenant in a public multi-tenant datacenter comprising forwarding elements to which the first network controller does not have access, the program comprising sets of instructions for:

identifying a first data compute node of the data compute nodes assigned to the particular tenant in the public multi-tenant datacenter that is configured to execute a second network controller; and

distributing configuration data defining the logical network to the first data compute node,

wherein the second network controller executing at the first data compute node distributes sets of the configuration data to a plurality of local agents executing on additional data compute nodes of the data compute nodes assigned to the particular tenant in the public multi-tenant datacenter that send and receive messages through the logical network, wherein both a managed forwarding element and a local agent execute on each of the additional data compute nodes, each local agent on a particular data compute node of the additional data compute nodes for receiving one of the sets of configuration data from the second network controller and configuring the managed forwarding element on the particular data compute node to implement the logical network according to the received set of configuration data.

12. The non-transitory machine readable medium of claim 11, wherein the program further comprises a set of instructions for distributing the configuration data to a plurality of local controller agents operating in virtualization software of host machines of a private datacenter operated by the particular tenant of the public multi-tenant datacenter, wherein the first network controller operates in the private datacenter.

13. The non-transitory machine readable medium of claim 12, wherein the logical network comprises one or more logical switches, each logical switch having a plurality of logical ports to which data compute nodes of the logical network connect, wherein the first network controller maps (i) each of the local controller agents in the private datacenter to one or more of the logical ports based on the data compute nodes operating on the host machine of the local controller agent and (ii) the second network controller to all of the logical ports the data compute nodes for which operate in the public multi-tenant datacenter.

14. The non-transitory machine readable medium of claim 11, wherein the program further comprises a set of instructions for receiving data regarding the additional data compute nodes via the second network controller executing on the first data compute node.

15. The non-transitory machine readable medium of claim 11, wherein the second network controller receives the distributed configuration data as a plurality of atomic pieces of data, identifies a span for each atomic piece of data, and distributes sets of the atomic pieces of data to the local agents based on the identified spans.

16. The non-transitory machine readable medium of claim 11, wherein the managed forwarding elements executing on the data compute nodes comprise flow-based forwarding elements, wherein the local agents receive configuration data from the second network controller and convert the received configuration data into flow entries for the forwarding elements.

17. The non-transitory machine readable medium of claim 11, wherein the first data compute node and the additional data compute nodes operate in a virtual private cloud of the public multi-tenant datacenter.

18. The non-transitory machine readable medium of claim 17, wherein the virtual private cloud is a first virtual private cloud, wherein the program further comprises sets of instructions for:

identifying a second data compute node of the data compute nodes assigned to the particular tenant in a second virtual private cloud of the public multi-tenant datacenter, the second data compute node configured to execute a third network controller; and

distributing the configuration data defining the logical network to the second data compute node,

wherein the third network controller executing at the second data compute node distributes sets of the configuration data to a second plurality of local agents executing on a second set of additional data compute nodes of the data compute nodes assigned to the particular tenant in the second virtual private cloud of the public multi-tenant datacenter that send and receive messages through the logical network, wherein both a managed forwarding element and a local agent execute on each of the additional data compute nodes in the second virtual private cloud, each local agent on a particular data compute node of the second set of additional data compute nodes for receiving one of the sets of configuration data from the third network controller and configuring the managed forwarding element on the particular data compute node to implement the logical network according to the received set of configuration data.

19. The non-transitory machine readable medium of claim 18, wherein the additional data compute nodes in the first virtual private cloud and the additional data compute nodes in the second virtual private cloud send and receive messages with each other through the logical network.

20. The non-transitory machine readable medium of claim 18, wherein the configuration data distributed to the first data compute node is different from the configuration data distributed to the second data compute node and comprises data that overlaps with the configuration data distributed to the second data compute node.