System and method for receive queue provisioning

• US 8,031,729 B2
• Filed: 09/30/2008
• Issued: 10/04/2011
• Est. Priority Date: 10/18/2002
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Elzur, Uri
• Primary Examiner(s): KAMEN, NOAH P
• Application Number:
  US12/241,917
• Publication Number:
  US 20090034551A1
• Time in Patent Office: 1,099 Days
• Field of Search: None
• US Class Current: 370/412
• CPC Class Codes: H04L 47/621 Individual queue per connection or flow, e.g. per VC
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 49/9036 Common buffer combined with individual queues
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/32 Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level

Abstract

Systems and methods that provide receive queue provisioning are provided. In one embodiment, a communications system may include, for example, a first queue pair (QP), a second QP, a general pool and a resource manager. The first QP may be associated with a first connection and with at least one of a first limit value and an out-of-order threshold. The first QP may include, for example, a first send queue (SQ). The second QP may be associated with a second connection and with a second limit value. The second QP may include, for example, a second SQ. The general pool may include, for example, a shared receive queue (SRQ) that is shared by the first QP and the second QP. The resource manager may provide, for example, provisioning for the SRQ and may manage the first limit value and the second limit value.

Claims

1. A local network node, comprising:

a local consumer operatively coupled to a network interface card (NIC) interface via a verb interface,

wherein the NIC interface is operatively coupled to a network that connects the local consumer with remote consumers at remote network nodes using multiple connections,

wherein the consumer communicates with a particular remote consumer at a particular remote network node using a first connection and a second connection,

wherein the NIC interface provides

a first queue pair (QP) associated with the first connection,

a second QP associated with the second connection, and

a shared receive queue (SRQ) shared by the first QP and the second QP,

wherein a watermark indicates a low level of work queue elements (WQEs) on the SRQ, and

wherein an out-of-order WQE limit is associated with the SRQ.

2. The local network node according to claim 1, wherein, if a number of WQEs on the SRQ drops below the watermark, then an asynchronous event is generated.

3. The local network node according to claim 2, wherein the asynchronous event notifies at least one of an upper layer protocol (ULP) layer, a buffer manager, and the local consumer that the number of WQEs on the SRQ has dropped below the watermark.

4. The local network node according to claim 2, wherein at least one of an upper layer protocol (ULP) layer, a buffer manager, and the local consumer analyzes completions posted on a shared completion queue (SCQ) shared by the first QP and the second QP.

5. The local network node according to claim 4, wherein the ULP layer determines which of the first QP or the second QP is transporting excessive traffic or offending traffic.

6. The local network node according to claim 1, wherein resources of the SRQ are dynamically allocated based on statistical information and actual demand information about the first connection or the second connection.

7. The local network node according to claim 1, wherein the local consumer provides a manager that provides statistical provisioning for the SRQ.

8. The local network node according to claim 7,

wherein an incoming message is received by the first QP,

wherein at least one of the first QP and the manager determines a number of resources to request for the first QP, and

wherein at least one of the first QP and the resource manager determines whether the first QP is allowed to draw any resources from the SRQ.

9. The local network node according to claim 1, wherein the local consumer monitors buffer usage by at least monitoring completions posted on a completion queue provided by the NIC interface.

10. The local network node according to claim 9, wherein the local consumer replenishes respective QP local limit values of the first QP and the second QP based upon at least the monitoring of the completions posted on the completion queue.

11. A local network node, comprising:

a local consumer operatively coupled to a network interface card (NIC) interface via a verb interface,

wherein the NIC interface is operatively coupled to a network that connects the local consumer with remote consumers at remote network nodes using multiple connections,

wherein the consumer communicates with a particular remote consumer at a particular remote network node using a first connection and a second connection,

wherein the NIC interface provides

a first queue pair (QP) associated with the first connection,

a second QP associated with the second connection, and

a shared receive queue (SRQ) shared by the first QP and the second QP,

wherein a watermark indicates a low level of work queue elements (WQEs) on the SRQ,

wherein a first out-of-order WQE limit is associated with the first QP, and

wherein a second out-of-order WQE limit is associated with the second QP.

12. The local network node according to claim 11, wherein resources of the SRQ are dynamically allocated based on statistical information and actual demand information about the first connection or the second connection.

13. The local network node according to claim 11, wherein, if a number of WQEs on the SRQ drops below the watermark, then an asynchronous event is generated.

14. The local network node according to claim 13, wherein the asynchronous event notifies at least one of an upper layer protocol (ULP) layer, a buffer manager, and the local consumer that the number of WQEs on the SRQ has dropped below the watermark.

15. The local network node according to claim 13, wherein at least one of an upper layer protocol (ULP) layer, a buffer manager, and any part of a consumer analyzes completions posted on a shared completion queue (SCQ) shared by the first QP and the second QP.

16. The local network node according to claim 15, wherein the ULP layer determines which of the first QP or the second QP is transporting excessive traffic or offending traffic.

17. The local network node according to claim 11, wherein the local consumer provides a manager that provides statistical provisioning for the SRQ.

18. The local network node according to claim 17,

wherein an incoming message is received by the first QP,

wherein at least one of the first QP and the manager determines a number of resources to request for the first QP, and

wherein at least one of the first QP and the resource manager determines whether the first QP is allowed to draw any resources from the SRQ.

19. The local network node according to claim 11, wherein the local consumer monitors buffer usage by at least monitoring completions posted on a completion queue provided by the NIC interface.

20. The local network node according to claim 19, wherein the local consumer replenishes respective QP local limit values of the first QP and the second QP based upon at least the monitoring of the completions posted on the completion queue.