Distributed load balancer node architecture

• US 9,432,245 B1
• Filed: 04/16/2013
• Issued: 08/30/2016
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Srinivasan, Venkatraghavan, Sorenson, James Christopher III, Laurence, Douglas Stewart
• Primary Examiner(s): Jakovac, Ryan
• Application Number:
  US13/864,152
• Time in Patent Office: 1,232 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04L 45/24 Multipath
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related

Abstract

A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. Each LB node may serve in ingress, egress, and/or flow tracker roles. Each LB node may include a first network interface controller (NIC) that faces the router and a second NIC that faces the server nodes. Each LB node may implement a core packet processing architecture in which packets received at the NICs are distributed among non-blocking input queues of a set of worker cores by receiver cores for the NICs, processed from the input queues by the worker cores, and placed on non-blocking input queues of transmit cores for the NICs.

Claims

1. A device, comprising;

a first network interface controller (NIC) configured to receive packets from and transmit packets onto a first network;

a second NIC configured to receive packets from and transmit packets onto a second network; and

a multicore packet processor comprising;

a first receive core configured to receive packets from the first NIC and, for each received packet, output the packet to one of a plurality of output queues of the first receive core according to a hash function applied to source and destination address information of the packet;

a second receive core configured to receive packets from the second NIC and, for each received packet, output the packet to one of a plurality of output queues of the second receive core according to a hash function applied to source and destination address information of the packet;

a first transmit core configured to output processed packets to the first NIC, the first transmit core coupled to a respective plurality of input queues to the first transmit core;

a second transmit core configured to output processed packets to the second NIC, the second transmit core coupled to a respective plurality of input queues to the second transmit core; and

a plurality of worker cores each coupled to a respective one of the plurality of output queues of the first receive core and to a respective one of the plurality of output queues of the second receive core, each worker core also coupled to a respective one of the plurality of input queues of the first transmit core and to a respective one of the plurality of input queues of the second transmit core, wherein each worker core is configured to;

read packets from its respective ones of the output queues of the first receive core and the second receive core;

for each packet, determine if the packet is to be accepted for processing; and

for each accepted packet;

perform one or more processing tasks on the packet;

determine, according to a protocol of the packet, that either the first transmit core or the second transmit core is to output the packet; and

output the processed packet to the respective input queue of the determined transmit core.

2. The device as recited in claim 1, wherein the device is configured to function as one of a plurality of load balancer nodes in a distributed load balancer system.

3. The device as recited in claim 2, wherein the device further comprises a third NIC configured to receive packets from and transmit packets onto a control plane of the distributed load balancer system according to Transmission Control Protocol (TCP) technology.

4. The device as recited in claim 2, wherein the first network is a border network of the distributed load balancer system, wherein the second network is a production network of the distributed load balancer system,

wherein the first NIC is configured to receive incoming packets from the border network;

wherein the second NIC is configured to transmit the processed incoming packets onto the production network for delivery to one or more server nodes on the production network;

wherein the second NIC is further configured to receive outgoing packets from the one or more server nodes on the production network; and

wherein the first NIC is further configured to transmit the processed outgoing packets onto the border network.

5. The device as recited in claim 1, wherein the packets received from and transmitted onto the first network by the first NIC are Transmission Control Protocol (TCP) packets, and wherein the packets received from and transmitted onto the second network by the second NIC are User Datagram Protocol (UDP) packets.

6. The device as recited in claim 1, wherein one of the plurality of worker cores is also coupled to an input queue of each of the other worker cores, wherein the worker core is further configured to output at least some of the packets processed by the worker core to the input queues of the other worker cores.

7. The device as recited in claim 6, wherein the worker core is also coupled to an input queue for receiving packets from one or more processes on the device and an output queue to the one or more other processes, wherein the worker core is further configured to read and process packets placed on the input queue by the one or more other processes and output the processed packets read from the input queue to the output queue to the one or more other processes.

8. The multicore packet processor as recited in claim 6, wherein the packets output by the worker core to the input queues of the other worker cores include packets that contain state information for the other worker cores, wherein the other worker cores are configured to update stored state information according to the state information in the packets.

9. The device as recited in claim 1, further comprising a memory, wherein each worker core is assigned a separate, non-overlapping region of the memory for storing packets processed by the respective core, and wherein each entry on each queue is a pointer to a packet stored in a memory region assigned to a respective worker core.

10. A multicore packet processor, comprising:

one or more hardware receive cores each configured to receive packets from one of one or more network interface controllers (NICs), each NIC configured to receive packets from and transmit packets onto one of one or more networks, each receive core coupled to a respective plurality of output queues, each receive core configured to, for each received packet, output the packet to one of the plurality of output queues of the receive core according to a hash function applied to source and destination address information of the packet;

one or more hardware transmit cores each configured to output processed packets to one of the one or more NICs, each transmit core coupled to a plurality of input queues; and

a plurality of hardware worker cores each coupled to a respective one of the plurality of output queues of each of the one or more receive cores and each coupled to a respective one of the plurality of input queues of each of the one or more transmit cores, each worker core configured to;

perform one or more processing tasks on packets read from the respective output queues of the receive cores, wherein one of the one or more processing tasks determines if the packets are accepted by the worker core; and

for each accepted packet;

determine one of the one or more transmit cores according to a protocol of the packet; and

output the processed packet to the respective input queue of the determined transmit core.

11. The multicore packet processor as recited in claim 10, wherein one of the plurality of worker cores is also coupled to an input queue of each of the other worker cores, wherein the worker core is further configured to output at least some of the packets processed by the worker core to the input queues of the other worker cores.

12. The multicore packet processor as recited in claim 11, wherein the packets output by the worker core to the input queues of the other worker cores include packets that contain state information for the other worker cores, wherein the other worker cores are configured to update stored state information according to the state information in the packets.

13. The multicore packet processor as recited in claim 11, wherein the worker core is also coupled to an input queue for receiving packets from one or more processes, wherein the worker core is further configured to read and process packets placed on the input queue by the one or more other processes.

14. The multicore packet processor as recited in claim 12, wherein the worker core is further configured to output the processed packets read from the input queue to an output queue to the one or more other processes.

15. The multicore packet processor as recited in claim 10, further comprising a memory, wherein each worker core is assigned a separate, non-overlapping region of the memory for storing packets processed by the respective worker core, and wherein each entry on each queue is a pointer to a packet stored in a memory region assigned to a respective worker core.

16. The multicore packet processor as recited in claim 10, wherein the multicore packet processor is configured to:

receive Transmission Control Protocol (TCP) packets from and transmit TCP packets onto a first network coupled to a first NIC; and

receive User Datagram Protocol (UDP) packets from and transmit UDP packets onto a second network coupled to a second NIC.

17. A method, comprising:

receiving, at each of two or more receive cores, packets from a respective one of two or more network interface controllers (NICs), wherein at least two of the NICs face different networks;

determining, by each of the receive cores, a particular one of a plurality of worker cores for each packet according to a hash function applied to source and destination address information of the packet; and

outputting, by each of the receive cores, each packet onto one of a plurality of input queues to the determined worker core for the respective packet, wherein each receive core outputs packets onto a different one of the plurality of input queues to each worker core; and

processing, by each of the plurality of worker cores, packets from the plurality of input queues to the respective worker core, wherein said processing comprises, for at least one packet on each worker core;

determining that the packet is accepted by the worker core;

determining one of two or more transmit cores for the packet according to a protocol of the packet; and

outputting the packet onto one of a plurality of input queues to the determined transmit core;

wherein each worker core outputs packets onto a different one of the plurality of input queues to each transmit core; and

wherein each transit core reads packets from the respective plurality of input queues and writes the packets to a respective one of the two or more NICS.

18. The method as recited in claim 17, wherein each worker core is assigned a separate, non-overlapping region of memory for storing packets processed by the respective worker core, wherein each entry on each queue is a pointer to a packet stored in a memory region assigned to a respective worker core.

19. The method as recited in claim 17, wherein the different networks include a border network and a production network of a distributed load balancer system, the method further comprising:

receiving, by a first one of the two or more NICs, incoming packets from the border network;

transmitting, by a second one of the two or more NICs, the processed incoming packets onto the production network for delivery to one or more server nodes on the production network;

receiving, by the second NIC, outgoing packets from the one or more server nodes on the production network; and

transmitting, by the first NIC, the processed outgoing packets onto the border network.

20. The method as recited in claim 19, wherein the packets received from and transmitted onto the border network by the first NIC are Transmission Control Protocol (TCP) packets, and wherein the packets received from and transmitted onto the production network by the second NIC are User Datagram Protocol (UDP) packets.