DISTRIBUTION OF MESSAGES TO QUEUES IN A DISTRIBUTED COMPUTING ENVIRONMENT

• US 20180183895A1
• Filed: 12/26/2016
• Published: 06/28/2018
• Est. Priority Date: 12/26/2016
• Status: Active Grant

• Current Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Original Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Inventors: Shuler, Shahaf, Bloch, Noam, Itigin, Yossef
• Granted Patent: US 10,623,521 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2209/548 Queue
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

A network adapter includes a network interface and circuitry. The network interface is assigned a single network address in a communication network, and is configured to receive, from one or more other nodes over the communication network, messages that are destined for processing by multiple threads in one or more processing cores of a network node including the network adapter, but are nevertheless addressed to the single network address. The circuitry is configured to hold a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads, and to select for each message received via the network interface a respective RQ, by applying the distribution rule to the message.

Claims

1. A network adapter, comprising:

a network interface, which is assigned a single network address in a communication network, and which is configured to receive, from one or more other nodes over the communication network, messages that are destined for processing by multiple threads in one or more processing cores of a network node comprising the network adapter, but are nevertheless addressed to the single network address; and

circuitry, which is configured to hold a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads, and to select for each message received via the network interface a respective RQ, by applying the distribution rule to the message.

2. The network adapter according to claim 1, wherein the network interface is configured to receive the messages for performing at least a portion of a distributed application program using the threads.

3. The network adapter according to claim 1, wherein the threads are assigned to handle messages of respective RQs in accordance with a predefined threads-to-RQ mapping, and wherein the circuitry is configured to select a RQ for a given message independently of the threads-to-RQ mapping.

4. The network adapter according to claim 1, wherein each thread is assigned a single respective RQ, and wherein the circuitry is configured to notify a given thread of a message added to the respective RQ.

5. The network adapter according to claim 1, wherein the circuitry is configured to select a RQ for a given message based on a message source identifier in a header of the message.

6. The network adapter according to claim 1, wherein the circuitry is configured to select a given RQ based on an occupancy status of multiple RQs including the given RQ.

7. The network adapter according to claim 1, wherein the circuitry is configured to select a RQ for a given message depending on previous messages received via the network interface.

8. The network adapter according to claim 1, wherein the RQs reside in a memory external to the network adapter, and wherein the circuitry is configured to manage the RQs logically by receiving from the cores a notification of the RQs allocated in the memory.

9. The network adapter according to claim 1, wherein the circuitry is configured to apply the distribution rule to multiple received messages for dynamically balancing a work load among the RQs.

10. A method, comprising:

in a network adapter, which comprises a network interface that is assigned a single network address in a communication network, receiving messages from one or more other nodes over the communication network, wherein the messages are destined for processing by multiple threads in one or more processing cores of the network node but are nevertheless addressed to the single network address;

holding a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads; and

selecting for each message received via the network interface a respective RQ, by applying the distribution rule to the message.

11. The method according to claim 10, wherein receiving the messages comprises receiving the messages for performing at least a portion of a distributed application program using the threads.

12. The method according to claim 10, wherein the threads are assigned to handle messages of respective RQs in accordance with a predefined threads-to-RQ mapping, and wherein applying the distribution rule comprises selecting the respective RQ independently of the threads-to-RQ mapping.

13. The method according to claim 10, wherein each thread is assigned a single respective RQ, and wherein the method comprises notifying a given thread of a message added to the respective RQ.

14. The method according to claim 10, wherein applying the distribution rule comprises selecting a RQ for a given message based on a message source identifier in a header of the message.

15. The method according to claim 10, wherein applying the distribution rule comprises selecting a given RQ based on an occupancy status of multiple RQs including the given RQ.

16. The method according to claim 10, wherein applying the distribution rule comprises selecting a RQ for a given message depending on previous messages received via the network interface.

17. The method according to claim 10, wherein the RQs reside in a memory external to the network adapter, and comprising managing the RQs logically by receiving from the cores a notification of the RQs allocated in the external memory.

18. The method according to claim 10, and comprising applying the distribution rule to multiple received messages for dynamically balancing a work load among the RQs.

19. A network node, comprising:

one or more processing cores, which are configured to execute multiple threads; and

a network adapter, which is configured to;

connect to a communication network via a network interface, which is assigned a single network address in the communication network;

receive from one or more other nodes over the communication network, via the network interface, messages that are destined for processing by the multiple threads, but are nevertheless addressed to the single network address;

hold a distribution rule for distributing the messages among multiple Receive Queues (RQs) that are accessible by the threads; and

select for each message received via the network interface a respective RQ, by applying the distribution rule to the message.

20. A computing system, comprising multiple network nodes connected to a communication network, each network node comprising one or more processing cores for executing multiple threads, wherein the network nodes are configured to:

exchange messages with one another over the communication network for performing a distributed application; and

in a given network node;

receive from one or more other nodes over the communication network, via a network interface for which a single network address is assigned in the communication network, messages that are destined for processing by multiple threads of the given network node, but are nevertheless addressed to the single network address;

hold a distribution rule for distributing the messages among Receive Queues (RQs) that are accessible by the threads of the given network node;

select for each message received via the network interface of the given network node, a respective RQ, by applying the distribution rule to the message; and

perform at least a portion of the distributed application using the threads.