Receive queue device with efficient queue flow control, segment placement and virtualization mechanisms

• US 20040049580A1
• Filed: 09/05/2002
• Published: 03/11/2004
• Est. Priority Date: 09/05/2002
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Recio, Renato John, Calvignac, Jean L., Boyd, William Todd, Chang, Chih-Jen, Joseph, Douglas J.
• Application Number:
  US10/235,672
• Publication Number:
  US 20040049580A1
• Time in Patent Office: Days
• US Class Current: 709/226
• CPC Class Codes: H04L 69/12 Protocol engines
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields

Abstract

A mechanism for offloading the management of receive queues in a split (e.g. split socket, split iSCSI, split DAFS) stack environment, including efficient queue flow control and TCP/IP retransmission support. An Upper Layer Protocol (ULP) creates receive work queues and completion queues that are utilized by an Internet Protocol Suite Offload Engine (IPSOE) and the ULP to transfer information and carry out send operations. As consumers initiate receive operations, receive work queue entries (RWQEs) are created by the ULP and written to the receive work queue (RWQ). The ISPOE is notified of a new entry to the RWQ and it subsequently reads this entry that contains pointers to the data that is to be received. After the data is received, the IPSOE creates a completion queue entry (CQE) that is written into the completion queue (CQ). After the CQE is written, the ULP subsequently processes the entry and removes it from the CQE, freeing up a space in both the RWQ and CQ. The number of entries available in the RWQ are monitored by the ULP so that it does not overwrite any valid entries. Likewise, the IPSOE monitors the number of entries available in the CQ, so as not overwrite the CQ.

Claims

1. A method, in a data processing system, for managing a work queue, comprising:

receiving a work request;

creating a receive work queue entry in a receive work queue in response to receipt of the work request;

sending a notification to an Internet Protocol Suite Offload Engine (IPSOE) notifying the IPSOE of the creation of the receive work queue entry; and

processing a completion queue entry in a completion queue in response to receiving a notification that the completion queue entry has been created by the IPSOE in response to processing of the receive work queue entry.

2. The method of claim 1, wherein the completion queue entry includes a work request ID that associates the receive work queue entry to the completion queue entry.

3. The method of claim 1, further comprising:

monitoring a number of entries available in the receive work queue so that valid receive work queue entries are not overwritten.

4. The method of claim 1, wherein the receive work queue entry is created by an upper layer protocol library in response to a receiving the work request.

28. The method of claim 1, further comprising:

providing a Selective Acknowledgment (SACK) table for storing information regarding incoming data packets destined for a receive queue of the IPSOE and for generating TCP/IP SACKs.

5. A method of sharing an Internet Protocol Suite Offload Engine (IPSOE) between virtual servers of a data processing system, comprising:

assigning a first address to a port of the IPSOE; and

assigning a second address to the port of the IPSOE, wherein the first address is assigned to a first virtual server, and wherein the second address is assigned to a second virtual server different from the first virtual server.

6. The method of claim 5, further comprising storing the first and second addresses for the port in a table.

7. The method of claim 5, further comprising:

receiving an incoming frame directed to the port, the incoming frame including a address;

performing a lookup in a address table associated with the IPSOE based on the address of the incoming frame; and

validating that the address of the incoming frame is one of the first and second addresses associated with the port.

8. The method of claim 6, wherein each entry in the table is assigned to a specific virtual server.

9. The method of claim 6, further comprising:

assigning an entry in the table for the port to a Send and Receive Queue Pair that access the port.

10. The method of claim 5, wherein the first and second addresses are Media Access Control (MAC) addresses.

11. The method of claim 5, wherein the first and second addresses are Internet Protocol (IP) aliases.

12. The method of claim 5, wherein the first virtual server is associated with a hosted server running an operating system instance on the data processing system, and wherein the second virtual server is associated with a hosting partition of the data processing system.

13. The method of claim 12, further comprising:

generating a software queue pair between the hosted server and the hosting partition.

14. The method of claim 13, wherein the hosted server places operations on a send queue of the queue pair, and wherein the hosting partition returns results of the operations to the hosted server through a receive queue of the queue pair.

15. The method of claim 13, further comprising:

generating a completion queue for the queue pair; and

generating a memory translation and protection table.

16. The method of claim 15, further comprising:

generating a server domain for associating at least one of the queue pair, completion queue and memory translation and protection table with the hosted server.

17. The method of claim 5, further comprising:

generating a first server domain for the first virtual server; and

generating a second server domain for the second virtual server, wherein the first server domain associates the first virtual server with a first set of IPSOE resources, and wherein the second server domain associates the second virtual server with a second set of IPSOE resources.

18. The method of claim 17, further comprising storing the first server domain in a queue pair context, a completion queue context and a Memory Translation and Protection Table entry associated with the first virtual server.

19. The method of claim 17, further comprising:

validating, for an incoming RDMA, that a Memory Translation and Protection Table entry associated with the incoming RDMA contains a same server domain as a queue pair context associated with the incoming RDMA.

20. The method of claim 17, further comprising:

validating, for a receive operation of a receive queue, that a Memory Translation and Protection Table entry associated with the receive operation contains a same server domain value as a queue pair context associated with the receive queue.

21. The method of claim 17, further comprising:

validating, for any of a Send, RDMA Write, or Bind operation of a send queue, that a Memory Translation and Protection Table entry associated with the operation contains a same server domain value as a queue pair context associated with the send queue.

22. The method of claim 17, further comprising:

validating, for completed work queue entries, that a completion queue context and queue pair that completed the work queue entry contain a same server domain value.

23. The method of claim 5, further comprising:

providing a IPSOE Resource Management Table (RMT) for assigning IPSOE resources to the first and second virtual servers.

24. The method of claim 23, further comprising:

storing in the RMT, for each physical IPSOE resource, a maximum value of the physical IPSOE resource.

25. The method of claim 23, further comprising:

storing in the RMT, for each variable physical IPSOE resource of the physical IPSOE resources, a total value of the variable physical IPSOE resource that has already been allocated.

26. The method of claim 23, further comprising:

storing, for each of the first and second virtual servers, a maximum amount of variable IPSOE resources, an amount of a fixed IPSOE resource that are assigned to the virtual server, and a total number of variable IPSOE resources that are currently in use by the virtual server.

27. The method of claim 26, further comprising:

providing, for each of the first and second virtual server, a mechanism for preventing the total number of variable IPSOE resources that are currently in use by the virtual server from exceeding the maximum variable IPSOE resources that are assigned to the virtual server.

29. A computer program product in a computer readable medium for sharing an Internet Protocol Suite Offload Engine (IPSOE) between virtual servers of a data processing system, comprising:

first instructions for assigning a first address to a port of the IPSOE; and

second instructions for assigning a second address to the port of the IPSOE, wherein the first address is assigned to a first virtual server, and wherein the second address is assigned to a second virtual server different from the first virtual server.

30. An apparatus for sharing an Internet Protocol Suite Offload Engine (IPSOE) between virtual servers of a data processing system, comprising:

means for assigning a first address to a port of the IPSOE; and

means for assigning a second address to the port of the IPSOE, wherein the first address is assigned to a first virtual server, and wherein the second address is assigned to a second virtual server different from the first virtual server.