Automatic configuration of logical routers on edge nodes

• US 10,333,849 B2
• Filed: 01/30/2017
• Issued: 06/25/2019
• Est. Priority Date: 04/28/2016
• Status: Active Grant

• Current Assignee: Nicira Incorporated (Broadcom, Inc.)
• Original Assignee: Nicira Incorporated (Broadcom, Inc.)
• Inventors: Masurekar, Uday, Goliya, Abhishek, Agarwal, Minjal
• Primary Examiner(s): O Connor, Brian T
• Application Number:
  US15/420,043
• Publication Number:
  US 20170317954A1
• Time in Patent Office: 876 Days
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 45/04 Interdomain routing, e.g. hierarchical routing
• CPC Class Codes: H04L 45/586 of virtual routers
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 45/745 Address table lookup; Address filtering
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 47/783 Distributed allocation of resources, e.g. bandwidth brokers
• CPC Class Codes: H04L 47/83 based on usage prediction
• CPC Class Codes: H04L 63/0218 Distributed architectures, e.g. distributed firewalls
• CPC Class Codes: H04L 63/0227 Filtering policies mail message filtering H04L51/212
• CPC Class Codes: H04L 63/0263 Rule management

Abstract

Some embodiments provide a method or tool for automatically configuring a logical router on one or more edge nodes of an edge cluster (e.g., in a hosting system such as a datacenter). The method of some embodiments configures the logical router on the edge nodes based on a configuration policy that dictates the selection method of the edge nodes. In some embodiments, an edge cluster includes several edge nodes (e.g., gateway machines), through which one or more logical networks connect to external networks (e.g., external logical and/or physical networks). In some embodiments, the configured logical router connects a logical network to an external network through the edge nodes.

Claims

1. A method for configuring a logical router for a logical network in a multi-tenant datacenter hosting a plurality of tenant logical networks, at least one tenant logical network spanning a plurality of host machines each hosting a data compute node (DCN) belonging to the tenant, the method comprising:

receiving configuration data for a logical router for a particular tenant logical network spanning a plurality of DCNs of the datacenter belonging to a particular tenant, wherein the logical router comprises a distributed component that performs routing for data messages sent to and from the DCNs of the particular logical network and a set of centralized components that perform stateful services for data messages sent to and from the DCNs of the particular logical network;

identifying an edge cluster comprising a plurality of candidate edge nodes for connecting the particular logical network to one or more external networks;

according to a set of selection rules configured for the logical router that use real-time statistical usage data for the plurality of candidate edge nodes, selecting one or more of the candidate edge nodes to implement centralized components of the logical router; and

configuring the selected edge nodes to implement the centralized components of the logical router.

2. The method of claim 1, wherein the centralized components are service routing components, wherein each service routing component of the logical router is configured on one selected edge node of the edge cluster.

3. The method of claim 2, wherein selecting one or more candidate edge nodes comprises selecting, for a particular service routing component, a first candidate edge node to implement an active service routing component and selecting a second candidate edge node to implement a standby routing component.

4. The method of claim 3, wherein the first candidate edge node is selected according to a first subset of the set of selection rules and the second candidate edge node is selected according to a second subset of the set of selection rules.

5. The method of claim 1 further comprising receiving a configuration policy from the particular tenant that comprises the set of selection rules and storing the configuration policy in a configuration policy database.

6. The method of claim 5, wherein selecting one or more candidate edge nodes according to a set of selection rules comprises identifying the set of selection rules configured for the logical router in the configuration policy database, wherein the configuration policy database comprises a section for each tenant logical network of the plurality of the tenant logical networks, the section comprising a set of selection rules for configuring one or more logical routers for the tenant logical network.

7. The method of claim 1, wherein

each rule in the set of selection rules has a priority order based on a rank that is assigned to the rule;

selecting one or more candidate edge nodes comprises (i) examining a particular selection rule in the set of selection rules, (ii) using the real-time statistical usage data to determine if one or more candidate edge nodes match the particular examined selection rule, and (iii) when one or more candidate edge nodes match the particular examined selection rule, selecting one of the matching candidate edge nodes; and

each selection rule with a higher rank is examined before examining any selection rule with a lower rank.

8. The method of claim 7 further comprising, when no edge node matches any rule in the set of selection rules, selecting a first available edge node for configuring the logical router.

9. The method of claim 1, wherein selecting one or more candidate edge nodes comprises:

from a configuration policy database, retrieving a rule for selecting an edge node that has a lowest number of connections to the external networks; and

examining a statistical usage database storing the real-time statistical usage data to select the candidate edge node with the least number of connections to the external network.

10. The method of claim 1, wherein selecting one or more candidate edge nodes comprises:

from a configuration policy database, retrieving a rule for selecting an edge node on which a lowest number of logical routers is configured; and

examining a statistical usage database storing the real-time statistical usage data to select the candidate edge node with the lowest number of configured logical routers.

11. The method of claim 1, wherein selecting one or more candidate edge nodes comprises:

from a configuration policy database, retrieving a rule for selecting an edge node that has been least recently used for logical router configuration; and

examining a statistical usage database storing the real-time statistical usage data to select the candidate edge node that has been least recently used for logical router configuration.

12. The method of claim 1, wherein selecting one or more candidate edge nodes comprises:

from a configuration policy database, retrieving a rule for selecting an edge node through which a least amount of network traffic passes; and

examining a statistical usage database storing the real-time statistical usage data to select the candidate edge node through which the least amount of network traffic has passed over a particular time interval.

13. The method of claim 1, wherein:

the logical router is a first logical router and the set of selection rules comprises at least one rule for selecting an edge node on which a centralized component of another logical router is configured when the other logical router connects the particular tenant logical network to one or more networks external to the multi-tenant datacenter; and

selecting one or more candidate edge nodes comprises selecting an edge node on which a centralized component of a second logical router is configured, the second logical router connecting the particular tenant logical network to a network external to the multi-tenant datacenter.

14. A non-transitory machine readable medium storing a program that when executed by at least one processing unit configures a logical router for a logical network in a multi-tenant datacenter hosting a plurality of tenant logical networks, at least one tenant logical network spanning a plurality of host machines each hosting a data compute node (DCN) belonging to the tenant, the program comprising sets of instructions for:

receiving configuration data for a logical router for a particular tenant logical network spanning a plurality of DCNs of the datacenter belonging to a particular tenant, wherein the logical router comprises a distributed component that performs routing for data messages sent to and from the DCNs of the particular logical network and a set of centralized components that perform stateful services for data messages sent to and from the DCNs of the particular logical network;

identifying an edge cluster comprising a plurality of candidate edge nodes for connecting the particular logical network to one or more external networks;

according to a set of selection rules configured for the logical router that use real-time statistical usage data for the plurality of edge nodes, selecting one or more of the candidate edge nodes to implement centralized components of the logical router; and

configuring the selected edge nodes to implement the centralized components of the logical router.

15. The non-transitory machine readable medium of claim 14, wherein the centralized components are service routing components, wherein the set of instructions for configuring the logical router comprises a set of instructions for automatically configuring each service routing component of the logical router on one selected edge node of the edge cluster.

16. The non-transitory machine readable medium of claim 14, wherein the selected edge nodes comprise a first set of edge nodes in the edge cluster, wherein the program further comprises a set of instructions for, before selecting the first set of edge nodes, identifying a second set of edge nodes in the edge cluster that are not qualified to implement the centralized components of the logical router.

17. The non-transitory machine readable medium of claim 16, wherein the edge nodes of the second set of edge nodes are not qualified based on at least one of a user defined constraint, a system constraint, and a product constraint.

18. The non-transitory machine readable medium of claim 14, wherein:

the logical router is a first logical router and the set of selection rules comprises at least one rule for selecting an edge node on which a centralized component of another logical router is configured; and

selecting one or more candidate edge nodes comprises selecting an edge node on which a centralized component of a second logical router is configured, the second logical router connecting the particular tenant logical network to a network external to the multi-tenant datacenter.

19. The non-transitory machine readable medium of claim 18, wherein

the first logical router is a tenant logical router;

the second logical router is a provider logical router for (i) performing routing for data messages sent to and from the DCNs of the particular tenant logical network and at least one other tenant logical network and (ii) connecting the particular tenant logical network and the other tenant logical network to the one or more external networks; and

the selection rule specifies that when a tenant logical router belongs to a same logical network to which a provider logical router belongs, the centralized components of the tenant logical router should be configured on the same candidate edge nodes of the edge cluster on which the centralized components of the provider logical router are configured.

20. The non-transitory machine readable medium of claim 18, wherein the program further comprises sets of instructions for:

receiving configuration data for a third logical router to be configured for the particular tenant logical network;

determining that the third logical router comprises only a distributed routing component and does not include any service routing component; and

foregoing configuring any of the edge nodes of the edge cluster to implement the third logical router.

21. The non-transitory machine readable medium of claim 14, wherein the program further comprises sets of instructions for:

receiving configuration data for a plurality of logical switches to be configured for the particular tenant logical network; and

configuring the plurality of logical switches on a plurality of host machines each of which hosts a set of DCNs, wherein the plurality of logical switches along with the logical router logically connect the sets of DCNs to each other and to an external network.

22. The non-transitory machine readable medium of claim 21, wherein each host machine of the plurality of host machines executes a managed forwarding element that implements the plurality of logical switches and the distributed routing component of the logical router by performing forwarding operations of each of the logical switches and of the distributed routing component.

23. The non-transitory machine readable medium of claim 22, wherein each edge node in the edge cluster also comprises a managed forwarding element that implements the plurality of logical switches and the distributed routing component of the logical router, wherein the host machines and edge nodes tunnel logical network traffic between each other through the managed forwarding elements.

24. The non-transitory machine readable medium of claim 14, wherein:

the program further comprises sets of instructions for receiving a configuration policy from the particular tenant that comprises the set of selection rules and storing the configuration policy in a configuration policy database;

the set of instructions for selecting one or more candidate edge nodes according to a set of selection rules comprise sets of instructions for identifying the set of selection rules configured for the logical router in the configuration policy database, wherein the configuration policy database comprises a section for each tenant logical network of the plurality of the tenant logical networks, the section comprising a set of selection rules for configuring one or more logical routers for the tenant logical network.