Distribution of messages to queues in a distributed computing environment

• US 10,623,521 B2
• Filed: 12/26/2016
• Issued: 04/14/2020
• Est. Priority Date: 12/26/2016
• Status: Active Grant

• Current Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Original Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Inventors: Shuler, Shahaf, Bloch, Noam, Itigin, Yossef
• Primary Examiner(s): Chea, Philip J
• Assistant Examiner(s): Khan, Hassan A
• Application Number:
  US15/390,558
• Publication Number:
  US 20180183895A1
• Time in Patent Office: 1,205 Days
• Field of Search: None
• CPC Class Codes: G06F 2209/548 Queue
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

A network adapter includes a network interface and circuitry. The network interface is assigned a single network address in a communication network, and is configured to receive, from one or more other nodes over the communication network, messages that are destined for processing by multiple threads in one or more processing cores of a network node including the network adapter, but are nevertheless addressed to the single network address. The circuitry is configured to hold a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads, and to select for each message received via the network interface a respective RQ, by applying the distribution rule to the message.

Claims

1. A network adapter, comprising:

a network interface, which is assigned a single network address in a communication network, and which is configured to receive, from one or more other nodes over the communication network, messages that are destined for processing by multiple threads in one or more processing cores of a network node comprising the network adapter, but are nevertheless addressed to the single network address, wherein each message comprises a header field comprising an identifier of a process running on a remote node that sent the message; and

circuitry, which is configured to;

hold a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads, wherein the distribution rule specifies a process-to-RQ mapping that maps one or more processes running on one or more remote nodes, by their respective identifiers, to each of the RQs;

receive a multi-packet message via the network interface;

extract from the header field of the received message a respective process identifier; and

select for the packets of the received message a respective RQ by mapping the extracted process identifier to the respective RQ using the process-to-RQ mapping.

2. The network adapter according to claim 1, wherein the network interface is configured to receive the messages for performing at least a portion of a distributed application program using the threads.

3. The network adapter according to claim 1, wherein the threads are assigned to handle messages of respective RQs in accordance with a predefined threads-to-RQ mapping, and wherein the circuitry is configured to select a RQ for a given message independently of the threads-to-RQ mapping.

4. The network adapter according to claim 1, wherein each thread is assigned a single respective RQ, and wherein the circuitry is configured to notify a given thread of a message added to the respective RQ.

5. The network adapter according to claim 1, wherein the circuitry is configured to select a given RQ based on an occupancy status of multiple RQs including the given RQ.

6. The network adapter according to claim 1, wherein the circuitry is configured to select a RQ for a given message depending on previous messages received via the network interface.

7. The network adapter according to claim 1, wherein the RQs reside in a memory external to the network adapter, and wherein the circuitry is configured to manage the RQs logically by receiving from the cores a notification of the RQs allocated in the memory.

8. The network adapter according to claim 1, wherein the circuitry is configured to apply the distribution rule to multiple received messages for dynamically balancing a work load among the RQs.

9. A method, comprising:

in a network adapter, which comprises a network interface that is assigned a single network address in a communication network, receiving messages from one or more other nodes over the communication network, wherein the messages are destined for processing by multiple threads in one or more processing cores of the network node but are nevertheless addressed to the single network address, and wherein each message comprises a header field comprising an identifier of a process running on a remote node that sent the message;

holding a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads, wherein the distribution rule specifies a process-to-RQ mapping that maps one or more processes running on one or more remote nodes, by their respective identifiers, to each of the RQs;

receiving a multi-packet message via the network interface;

extracting from the header field of the received message a respective process identifier; and

selecting for the packets of the received message a respective RQ by mapping the extracted process identifier to the respective RQ using the process-to-RQ mapping.

10. The method according to claim 9, wherein receiving the messages comprises receiving the messages for performing at least a portion of a distributed application program using the threads.

11. The method according to claim 9, wherein the threads are assigned to handle messages of respective RQs in accordance with a predefined threads-to-RQ mapping, and wherein applying the distribution rule comprises selecting the respective RQ independently of the threads-to-RQ mapping.

12. The method according to claim 9, wherein each thread is assigned a single respective RQ, and wherein the method comprises notifying a given thread of a message added to the respective RQ.

13. The method according to claim 9, wherein applying the distribution rule comprises selecting a given RQ based on an occupancy status of multiple RQs including the given RQ.

14. The method according to claim 9, wherein applying the distribution rule comprises selecting a RQ for a given message depending on previous messages received via the network interface.

15. The method according to claim 9, wherein the RQs reside in a memory external to the network adapter, and comprising managing the RQs logically by receiving from the cores a notification of the RQs allocated in the external memory.

16. The method according to claim 9, and comprising applying the distribution rule to multiple received messages for dynamically balancing a work load among the RQs.

17. A network node, comprising:

one or more processing cores, which are configured to execute multiple threads; and

a network adapter, which is configured to;

connect to a communication network via a network interface, which is assigned a single network address in the communication network;

receive from one or more other nodes over the communication network, via the network interface, messages that are destined for processing by the multiple threads, but are nevertheless addressed to the single network address, wherein each message comprises a header field comprising an identifier of a process running on a remote node that sent the message;

hold a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads, wherein the distribution rule specifies a process-to-RQ mapping that maps one or more processes running on one or more remote nodes, by their respective identifiers, to each of the RQs;

receive a multi-packet message via the network interface;

extract from the header field of the received message a respective process identifier; and

select for the packets of the received message a respective RQ by mapping the extracted process identifier to the respective RQ using the process-to-RQ mapping.

18. A computing system, comprising multiple network nodes connected to a communication network, each network node comprising one or more processing cores for executing multiple threads, wherein the network nodes are configured to:

exchange messages with one another over the communication network for performing a distributed application; and

in a given network node;

receive from one or more other nodes over the communication network, via a network interface for which a single network address is assigned in the communication network, messages that are destined for processing by multiple threads of the given network node, but are nevertheless addressed to the single network address;

hold a distribution rule for distributing the messages among Receive Queues (RQs) that are accessible by the threads of the given network node, wherein the distribution rule specifies a process-to-RQ mapping that maps one or more processes running on one or more remote nodes, by their respective identifiers, to each of the RQs;

receive a multi-packet message via the network interface;

extract from the header field of the received message a respective process identifier;

select for the packets of the received message a respective RQ by mapping the extracted process identifier to the respective RQ using the process-to-RQ mapping; and

perform at least a portion of the distributed application using the threads.