Unified control plane for nested clusters in a virtualized computing infrastructure

• US 10,708,082 B1
• Filed: 08/31/2018
• Issued: 07/07/2020
• Est. Priority Date: 08/31/2018
• Status: Active Grant

• Current Assignee: Juniper Networks Incorporated
• Original Assignee: Juniper Networks Incorporated
• Inventors: Bakiaraj, Dinesh, Mariappan, Yuvaraja, Vaidya, Sachchidanand, Rugge, Rudramahesh S.
• Primary Examiner(s): Cho, Un C
• Assistant Examiner(s): Rahman, Shah M
• Application Number:
  US16/118,731
• Time in Patent Office: 676 Days
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: H04L 12/4625 Single bridge functionality, e.g. connection of two networks over a single bridge
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 12/4666 Operational details on the addition or the stripping of a tag in a frame, e.g. at a provider edge node
• CPC Class Codes: H04L 41/0806 for initial configuration or provisioning, e.g. plug-and-play
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 41/5051 Service on demand, e.g. definition and deployment of services in real time
• CPC Class Codes: H04L 45/586 of virtual routers
• CPC Class Codes: H04L 45/64 using an overlay routing layer

Abstract

Techniques are disclosed for a unified control plane in a nested cluster environment. In one example, an underlay orchestrator for a virtualized computing infrastructure is configured to provision, in an underlay cluster of one or more servers, an overlay cluster of one or more overlay nodes. Each of the overlay nodes is a workload of one of the servers and has a virtual network interface. Further, each server executes a virtual router and a virtual router agent for the virtual router for routing packets among virtual network interfaces of the overlay nodes executed by the server. A network controller is configured to configure virtual network sub-interfaces for workloads of the overlay nodes to enable communications among workloads executing in different overlay nodes. Each of the sub-interfaces is a virtual network sub-interface of one of the virtual network interfaces.

Claims

1. A controller comprising one or more computing devices, wherein each of the one or more computing devices comprises processing circuitry coupled to a memory device, wherein the controller further comprises:

an underlay orchestrator for a virtualized computing infrastructure, wherein the underlay orchestrator is configured for execution by the processing circuitry, and wherein the underlay orchestrator is configured to;

provision, in an underlay cluster of one or more servers, an overlay cluster of one or more overlay nodes, wherein each of the one or more overlay nodes is a workload of one of the one or more servers and has a virtual network interface,

wherein each server of the one or more servers executes a virtual router and a virtual router agent for the virtual router for routing packets among virtual network interfaces of the overlay nodes executed by the server; and

a network controller configured to;

receive, from a network controller manager of a master overlay node that orchestrates workloads of the one or more overlay nodes, a request to create virtual network sub-interfaces for workloads of the one or more overlay nodes; and

configure, in response to receiving the request from the network controller manager, virtual network sub-interfaces for the workloads of the one or more overlay nodes to enable communications among workloads executing in different overlay nodes, wherein each of the virtual network sub-interfaces is a sub-interface of one of the virtual network interfaces.

2. The controller of claim 1, wherein, to configure the sub-interfaces, the network controller is further configured to configure the virtual network interfaces with unique virtual network sub-interface identifiers for the virtual network sub-interfaces.

3. The controller of claim 2, wherein each of the unique sub-interface identifiers for the virtual network sub-interfaces comprises a unique Media Access Control (MAC)-based Virtual Local Area Network (VLAN) identifier for a VLAN of the one or more overlay nodes.

4. The controller of claim 1, wherein each of the virtual routers is configured to:

receive network traffic associated with the workloads, wherein the network traffic includes a virtual network interface identifier and a virtual network sub-interface identifier; and

forward, based on the virtual network interface identifier and the virtual network sub-interface, identifier, the network traffic to a workload of the one or more overlay nodes.

5. The controller of claim 4, wherein each virtual router is further configured to remove a first header of the network traffic that includes the virtual network interface identifier prior to forwarding the network traffic to a virtual network interface of the workload of the one or more overlay nodes that corresponds to the virtual network interface identifier.

6. The controller of claim 5, wherein each virtual router is further configured to:

remove a second header of the network traffic that includes the virtual network sub-interface identifier prior to forwarding the network traffic to a virtual network sub-interface for the workload of the one or more overlay nodes that corresponds to the virtual network sub-interface identifier.

7. The controller of claim 6,

wherein the virtual network interface identifier comprises a first virtual network interface identifier, the virtual network interface for the workload comprises a first virtual network interface for a first workload, the virtual network sub-interface identifier comprises a first virtual network sub-interface identifier, and the virtual network sub-interface comprises a first virtual network sub-interface for the first workload, and

wherein each virtual router is further configured to:

receive network traffic associated with the first workload; and

forward, based on a second virtual network interface identifier for a second virtual network interface for a second workload of the one or more overlay nodes and a second virtual network sub-interface identifier of a second virtual network sub interface of the second workload, the network traffic to a virtual router for the second virtual network.

8. The controller of claim 7, wherein each virtual router is further configured to attach a first header to the network traffic that includes the second virtual network interface identifier prior to forwarding the network traffic to the second virtual router.

9. The controller of claim 8, wherein each virtual router is further configured to attach a second header to the network traffic that includes the second virtual network sub-interface identifier prior to forwarding the network traffic to the second virtual router.

10. The controller of claim 1, wherein the network controller is further configured to invoke a network plugin of the one or more overlay nodes to configure the virtual network sub-interfaces for the workloads of the one or more overlay nodes.

11. The controller of claim 1,

wherein the master overlay node comprises an overlay orchestrator,

wherein the network controller is configured to receive, from the overlay orchestrator, one or more events, wherein the one or more events comprise at least one of a Create, Read, Update, or Delete (CRUD) request, a networking request, or a policy request.

12. The controller of claim 1, wherein the underlay cluster executes on one or more bare-metal servers, each of the one or more bare-metal servers comprising processing circuitry and a storage device.

13. A method comprising:

provisioning, by an underlay orchestrator of a controller for a virtualized computing infrastructure, an overlay cluster of one or more overlay nodes in an underlay cluster of one or more servers,

wherein each of the one or more overlay nodes is a workload of one of the one or more servers and has a virtual network interface,

wherein the underlay orchestrator is configured for execution by the processing circuitry, and

wherein each server of the one or more servers executes a virtual router and a virtual router agent for the virtual router for routing packets among virtual network interfaces of the overlay nodes executed by the server;

receive, by a network controller of the controller and from a network controller manager of a master overlay node that orchestrates workloads of the one or more overlay nodes, a request to create virtual network sub-interfaces for workloads of the one or more overlay nodes; and

configuring, by the network controller and in response to receiving the request from the network controller manager, virtual network sub-interfaces for the workloads of the one or more overlay nodes to enable communications among workloads executing in different overlay nodes, wherein each of the virtual network sub-interfaces is a sub-interface of one of the virtual network interfaces.

14. The method of claim 13, wherein configuring the virtual network sub-interfaces comprises configuring, by the network controller, the virtual network interfaces with unique virtual network sub-interface identifiers for the virtual network sub-interfaces.

15. The method of claim 14, wherein each of the unique sub-interface identifiers for the virtual network sub-interfaces comprises a unique Media Access Control (MAC)-based Virtual Local Area Network (VLAN) identifier for a VLAN of the one or more overlay nodes.

16. The method of claim 13, further comprising configuring, by the network controller, each virtual router to:

receive network traffic associated with the workloads, wherein the network traffic includes a virtual network interface identifier and a virtual network sub-interface identifier; and

forward, based on the virtual network interface identifier and the virtual network sub-interface identifier, the network traffic to a workload of the one or more overlay nodes.

17. The method of claim 13,

wherein the workload of the one or more overlay nodes comprises a first workload, the virtual network interface identifier comprises a first virtual network interface identifier that specifies a first virtual network interface for the first workload, and the virtual network sub-interface identifier comprises a first virtual network sub-interface identifier that specifies a first virtual network sub-interface for the first workload, and

wherein the method further comprises configuring, by the network controller, each virtual router to:

receive network traffic associated with the first workload; and

forward, based on a second virtual network interface identifier for a second virtual network interface for a second workload of the one or more overlay nodes and a second virtual network sub-interface identifier of a second virtual network sub-interface of the second workload, the network traffic to a second virtual router for the second virtual network.

18. The method of claim 13, wherein the underlay cluster executes on one or more bare-metal servers, each of the one or more bare-metal servers comprising processing circuitry and a storage device.

19. A system comprising:

a first computing device comprising processing circuitry coupled to a memory device,

wherein the first computing device executes one or more overlay minion nodes of an overlay cluster,

wherein each of the one or more overlay minion nodes has a virtual network interface and executes workloads,

wherein the first computing device further executes a virtual router and a virtual router agent for the virtual router for routing packets among virtual network interfaces of the one or more overlay minion nodes of the overlay cluster; and

a second computing device comprising processing circuitry coupled to a memory device,

wherein the second computing device is configured to execute an overlay orchestrator configured to orchestrate the workloads executed by the one or more overlay minion nodes of the overlay cluster,

wherein the second computing device is further configured to execute a network controller manager configured to cause a network controller to configure virtual network sub-interfaces for the workloads executed by the one or more overlay minion nodes of the overlay cluster, and

wherein the virtual router agent is configured to receive, from the network controller, network configuration data for the virtual network sub-interfaces for the workloads,

wherein the first computing device is further configured to execute a network module configured to;

obtain the network configuration data from the virtual router agent; and

based on the network configuration data, configure virtual network sub-interfaces for the workloads of the one or more overlay nodes to enable communications among workloads executing in different overlay nodes, wherein each of the virtual network sub-interfaces is a sub-interface of one of the virtual network interfaces.

20. The system of claim 19, wherein the network module is further configured to, based on the network configuration data, configure the virtual network interfaces of the one or more overlay minion nodes.