Protocol flow control

• US 20070162559A1
• Filed: 01/12/2006
• Published: 07/12/2007
• Est. Priority Date: 01/12/2006
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Biswas, Amitabha, Modi, Prashant
• Granted Patent: US 7,895,329 B2
• Time in Patent Office: Days
• US Class Current: 709/216
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/39 Credit based
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/6225 Fixed service order, e.g. Round Robin

Abstract

In a flow control method, operation flow is controlled on a per connection basis. The method comprises managing a sending peer to prevent overflow of a receiving peer'"'"'s receiver credits and allocating a number of send buffers at the sending peer greater than or equal to the maximum number of receiver credits that can be indicated by the receiving peer. The technique prevents the sending peer from overflowing its set of send buffers, arising due to re-use, without taking completions on operations queued to a Send Queue.

Claims

1. A Remote Direct Memory Access (RDMA) flow control method comprising:

controlling operation flow on a per connection level;

managing a sending peer to prevent overflow of a receiving peer'"'"'s receiver credits; and

allocating a number of send buffers at the sending peer greater than or equal to the maximum number of receiver credits that can be indicated by the receiving peer.

2. The method according to claim 1 further comprising:

tracking at the sending peer a number of receive credits available at the receiving peer; and

transferring from the sending peer to the receiving peer only in conditions that the transfer does not overflow the receiving peer'"'"'s posted receive credits.

3. The method according to claim 1 further comprising:

reusing a send buffer at the sending peer only in a condition that a previous transfer from the send buffer is completed.

4. The method according to claim 1 further comprising:

activating send buffers at the sending peer in an order selected from among a group comprising round robin, first-in-first-out (FIFO), and random order once the send buffer is validated as completed.

5. The method according to claim 1 further comprising:

sending a notification from the receiving peer to the sending peer that informs the sending peer about the number of receive credits available at the receiving peer, the notification identifying which send buffers at the sending peer are completed; and

optimizing flow control using the sent notification.

6. A Remote Direct Memory Access (RDMA) flow control apparatus comprising:

a node operative as a sending peer and adapted to control operation flow on a per connection level;

a plurality of send buffers at the node; and

a logic operative at the node adapted to manage the sending peer to prevent overflow of a receiving peer'"'"'s receiver credits and adapted to allocate a number of send buffers at the sending peer greater than or equal to the maximum number of receiver credits that can be indicated by the receiving peer.

7. The apparatus according to claim 6 further comprising:

a user software layer;

a network interface; and

the logic operative to control kernel bypass access to the network interface.

8. The apparatus according to claim 6 further comprising:

an application;

a protocol layer comprising a plurality of send buffers and a plurality of receive buffers, the protocol layer adapted to communicate with the application via a sockets applications programming interface (API);

a provider adapted to communicate with the protocol layer via at least one protocol applications programming interface (API);

a network interface comprising a send queue and a receive queue, the network interface adapted to communicate with the provider via a RDMA Network Interface API.

9. The apparatus according to claim 6 further comprising:

the logic adapted to track at the sending peer a number of receive credits available at the receiving peer and adapted to transfer from the sending peer to the receiving peer only in conditions that the transfer does not overflow the receiving peer'"'"'s posted receive credits.

10. The apparatus according to claim 6 further comprising:

the logic adapted to reuse a send buffer at the sending peer only in a condition that a previous transfer from the send buffer is completed.

11. The apparatus according to claim 6 further comprising:

the logic adapted to send a notification from the receiving peer to the sending peer that informs the sending peer about the number of receive credits available at the receiving peer, the notification identifying which send buffers at the sending peer are completed.

12. A network comprising:

a plurality of systems, at least one of the systems plurality adapted to execute a process; and

at least one node adapted to communicate a data packet from the at least one of the systems plurality, the at least one node operative as a sending peer and adapted to control operation flow on a per connection level, the at least one node further comprising;

a plurality of send buffers at the node; and

a logic operative at the node adapted to manage the sending peer to prevent overflow of a receiving peer'"'"'s receiver credits and adapted to allocate a number of send buffers at the sending peer greater than or equal to the maximum number of receiver credits that can be indicated by the receiving peer.

13. The network according to claim 12 wherein the at least one node further comprises:

a user software layer;

a network interface; and

the logic operative to control kernel bypass access to the network interface.

14. The network according to claim 12 wherein the at least one node further comprises:

an application;

a protocol layer comprising a plurality of send buffers and a plurality of receive buffers, the protocol layer adapted to communicate with the application via a sockets applications programming interface (API);

a provider adapted to communicate with the protocol layer via at least one protocol applications programming interface (API);

a network interface comprising a send queue and a receive queue, the network interface adapted to communicate with the provider via a RDMA Network Interface API.

15. The network according to claim 12 wherein the at least one node further comprises:

the logic adapted to track at the sending peer a number of receive credits available at the receiving peer and adapted to transfer from the

sending peer to the receiving peer only in conditions that the transfer does not overflow the receiving peer'"'"'s posted receive credits.

16. The network according to claim 12 wherein the at least one node further comprises:

the logic adapted to reuse a send buffer at the sending peer only in a condition that a previous transfer from the send buffer is completed.

17. The network according to claim 12 wherein the at least one node further comprises:

the logic adapted to send a notification from the receiving peer to the sending peer that informs the sending peer about the number of receive credits available at the receiving peer, the notification identifying which send buffers at the sending peer are completed.

18. An article of manufacture comprising:

a controller usable medium having a computable readable program code embodied therein adapted to control communication flow in a network, the computable readable program code further comprising;

a code adapted to cause the controller to control Remote Direct Memory Access (RDMA) operation flow on a per connection level;

a code adapted to cause the controller to manage a sending peer to prevent overflow of a receiving peer'"'"'s receiver credits; and

a code adapted to cause the controller to allocate a number of send buffers at the sending peer greater than or equal to the maximum number of receiver credits that can be indicated by the receiving peer.

19. The article of manufacture according to claim 18 wherein the computable readable program code further comprises:

a code adapted to cause the controller to track at the sending peer a number of receive credits available at the receiving peer; and

a code adapted to cause the controller to transfer from the sending peer to the receiving peer only in conditions that the transfer does not overflow the receiving peer'"'"'s posted receive credits.

20. The article of manufacture according to claim 18 wherein the computable readable program code further comprises:

a code adapted to cause the controller to reuse a send buffer at the sending peer only in a condition that a previous transfer from the send buffer is completed.