Network server having hardware-based virtual router integrated circuit for virtual networking

• US 10,382,362 B2
• Filed: 04/13/2018
• Issued: 08/13/2019
• Est. Priority Date: 03/31/2014
• Status: Active Grant

• Current Assignee: Juniper Networks Incorporated
• Original Assignee: Juniper Networks Incorporated
• Inventors: Sindhu, Pradeep
• Primary Examiner(s): Elpenord, Candal
• Application Number:
  US15/953,112
• Publication Number:
  US 20180241696A1
• Time in Patent Office: 487 Days
• Field of Search: None
• CPC Class Codes: H04L 12/4633 Interconnection of networks using encapsulation techniques, e.g. tunneling
• CPC Class Codes: H04L 45/64 using an overlay routing layer
• CPC Class Codes: H04L 47/6215 Individual queue per QOS, rate or priority
• CPC Class Codes: H04L 49/10 characterised by the switching fabric construction
• CPC Class Codes: H04L 49/111 Switch interfaces, e.g. port details
• CPC Class Codes: H04L 49/253 using establishment or release of connections between ports
• CPC Class Codes: H04L 49/70 Virtual switches

Abstract

A high-performance, scalable and drop-free data center switch fabric and infrastructure is described. The data center switch fabric may leverage low cost, off-the-shelf packet-based switching components (e.g., IP over Ethernet (IPoE)) and overlay forwarding technologies rather than proprietary switch fabric. In one example, host network accelerators (HNAs) are positioned between servers (e.g., virtual machines or dedicated servers) of the data center and an IPoE core network that provides point-to-point connectivity between the servers. The HNAs are hardware devices that embed virtual routers on one or more integrated circuits, where the virtual router are configured to extend the one or more virtual networks to the virtual machines and to seamlessly transport packets over the switch fabric using an overlay network. In other words, the HNAs provide hardware-based, seamless access interfaces to overlay technologies used for communicating packet flows through the core switching network of the data center.

Claims

1. A server comprising:

a multi-core processor having a plurality of processing cores;

a plurality of virtual machines configured to execute on the multi-core processor;

a physical network interface to connect the server to a switch fabric comprising a plurality of switches that provide switching for packets through a physical network, wherein the physical network interface is coupled to the multi-core processor by a data path provided by an input/output (I/O) interface; and

a hardware-based virtual router executing within one or more integrated circuits positioned on the data path of the I/O interface between the physical network interface and the multi-core processor, the virtual router configured to apply routing information of a plurality of virtual networks to route packets between the physical network interface and the virtual machines configured to execute on the multi-core processor,

wherein the virtual router is configured to receive outbound packets from the server by the I/O interface and construct outbound tunnel packets in accordance with an overlay network extending across the switch fabric, wherein the outbound tunnel packets encapsulate the outbound packets, and

wherein the virtual router is configured to receive inbound tunnel packets from the switch fabric by the physical network interface, extract inner packets encapsulated within the inbound tunnel packets and route the inner packets to the virtual machines via the I/O interface in accordance with routing information for the virtual networks.

2. The server of claim 1, further comprising:

a plurality of outbound queues to buffer the outbound tunnel packets output by the virtual router; and

a scheduler that controls transmission of the outbound tunnel packets from the virtual router, via the switch fabric, to virtual routers executing within other servers coupled to the switch fabric.

3. The server of claim 2,

wherein the plurality of outbound queues comprises a plurality of sets of output queues, each of the sets of output queues buffering a respective set of the outbound tunnel packets that are to be sent to a corresponding one of the other virtual routers,

wherein each of the outbound queues within each of the respective sets of outbound queues is assigned a different priority, and

wherein, for each of the respective sets of outbound queues, the scheduler controls transmission of the outbound tunnel packets based on the respective priorities assigned to the outbound queues.

4. The server of claim 2, further comprising a flow control unit that exchanges flow control information with each of a set of the other virtual routers coupled to the switch fabric and positioned between the switch fabric and the remote servers,

wherein the scheduler controls transmission of the outbound tunnel packets from the virtual router to each of the other ones of the virtual routers based on the respective flow control information received from the other ones of the host network accelerators.

5. The server of claim 4, wherein the flow control information sent by the flow control unit to each of the other virtual routers specifies a maximum rate at which the respective virtual routers to which the flow control information is being sent is permitted to send tunnel packets to the virtual router.

6. The server of claim 5, wherein the flow control unit inserts the flow control information within the outbound tunnel packets constructed by the virtual router when encapsulating outbound packets from the virtual machines.

7. The server of claim 5, wherein the flow control unit inserts the flow control information into an outer header of the outbound tunnel packets constructed by the virtual router when encapsulating outbound packets from the virtual machines.

8. The server of claim 5, wherein, for each of the other virtual routers, the flow control unit sends a heartbeat message carrying the flow control information when the virtual router has not sent outbound tunnel packets to the respective virtual routers over the switch fabric for a threshold period of time.

9. The server of claim 1, wherein the network interface communicates the tunnel packets to the switch fabric using Internet Protocol over Ethernet (IPoE) protocols.

10. The server of claim 1, wherein the server provides an operating environment executing one or more virtual machines associated with one or more of the virtual networks.

11. The server of claim 1, wherein the server comprises a storage server.

12. The server of claim 1, further comprising a memory for storing the outbound packets as directly written into the memory by a processor of the server.

13. The server of claim 1, wherein the PCIe interface comprises a PCIe bus.

14. A method comprising:

receiving, from a processor of a server and by an input/output (I/O) interface coupled to the processor, a plurality of outbound packets;

selecting, with a hardware-based virtual router executing on one or more integrated circuits of a network interface card, destinations within virtual networks for the outbound packets, wherein the one or more integrated circuits of the network interface card are positioned on a data path between the I/O interface and a physical network interface provided by the network interface card;

constructing, with the virtual router, outbound tunnel packets based on the selected destinations and in accordance with an overlay network extending across a physical network to a plurality of other virtual routers executing within network interface cards deployed within other servers coupled to the physical network, wherein the outbound tunnel packets encapsulate the outbound packets; and

forwarding, by the physical network interface of the network interface card, the outbound tunnel packets to the physical network.

15. The method of claim 14, further comprising:

receiving, with the network interface card, flow control information from the network interface cards deployed within the other servers coupled to the physical network; and

controlling, with the network interface card, transmission of the outbound tunnel packets from the virtual router to other ones of the network interface cards based on the respective flow control information received from each of the other ones of the network interface cards.

16. The method of claim 14, further comprising, prior to forwarding the outbound tunnel packets to the network interface, inserting, with the virtual router, flow control information into outer headers of the outbound tunnel packets.

17. The method of claim 14, further comprising:

sending, to each of the other network interface cards, flow control information specifying a maximum rate at which the respective other network interface card is permitted to send tunnel packets to the network interface card.

18. The method of claim 16, further comprising sending, from the network interface card and to one of the other network interface cards, a heartbeat message carrying the flow control information when the virtual router has not sent outbound tunnel packets to the other one of the network interface cards over the switch fabric for a threshold period of time.

19. The method of claim 14, wherein receiving the outbound packets comprises receiving outbound packets written directly to memory of the integrated circuits.