Infiniband work and completion queue management via head and tail circular buffers with indirect work queue entries

• US 20030061417A1
• Filed: 09/24/2001
• Published: 03/27/2003
• Est. Priority Date: 09/24/2001
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Judd, Ian David, Recio, Renato John, Craddock, David F., Schmidt, Donald William, Gregg, Thomas Anthony, Pfister, Gregory Francis
• Granted Patent: US 6,789,143 B2
• Time in Patent Office: Days
• US Class Current: 710/54
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues

Abstract

A distributed computing system having (host and I/O) end nodes, switches, routers, and links interconnecting these components is provided. The end nodes use send and receive queue pairs to transmit and receive messages. The end nodes use completion queues to inform the end user when a message has been completely sent or received and whether an error occurred during the message transmission or reception process. A mechanism implements these queue pairs and completion queues in hardware. A mechanism for controlling the transfer of work requests from the consumer to the CA hardware and work completions from the CA hardware to the consumer using head and tail pointers that reference circular buffers is also provided. The QPs and CQs do not contain Work Queue Entries and Completion Queue Entries respectively, but instead contain references to these entries. This allows them to be efficient and constant in size, while the Work Queue Entries and Completion Queue Entries themselves can vary in size, for example to include a variable number of data segments. Additionally, several mechanisms are provided to improve the overall efficiency of this process under different memory configurations.

Claims

1. A method for managing a work request, comprising:

building a work queue entry for a work queue;

writing the work queue entry in memory; and

writing, in the work queue, a work queue entry pointer that points to the work queue entry.

2. The method of claim 1, further comprising:

performing a comparison of a queue head index and a queue tail index incremented by one to determine whether work queue is full; and

writing the work queue entry if the work queue is not full.

3. The method of claim 1, wherein the work queue entry pointer comprises a real address.

4. The method of claim 1, wherein the work queue entry pointer comprises a virtual address.

5. The method of claim 1, wherein the step of writing a work queue entry pointer comprises writing the work queue entry pointer at a location referenced by a queue tail index.

6. A method for managing a work request, comprising:

determining a location of a work queue entry pointer;

reading a work queue entry referenced by the work queue entry pointer; and

transmitting a message corresponding to the work queue entry.

7. The method of claim 6, wherein the work queue entry pointer comprises a real address.

8. The method of claim 6, wherein the work queue entry pointer comprises a virtual address.

9. The method of claim 6, further comprising:

writing a completion queue entry corresponding to the work queue entry in memory.

10. The method of claim 9, further comprising writing a completion queue entry pointer that points to the completion queue entry in a completion queue.

11. The method of claim 10, wherein the step of writing a completion queue entry pointer comprises writing the completion queue entry pointer at a location referenced by a queue tail index.

12. A method for managing a request for work completion information, comprising:

reading a completion queue entry pointer;

reading a completion queue entry referenced by the completion queue entry pointer; and

using the completion queue entry to return work completion information.

13. The method of claim 12, wherein the completion queue entry pointer comprises a real address.

14. The method of claim 12, wherein the completion queue entry pointer comprises a virtual address.

15. A computer program product, in a computer readable medium, for managing a work request, comprising:

instructions for building a work queue entry for a work queue;

instructions for writing the work queue entry in memory; and

instructions for writing, in the work queue, a work queue entry pointer that points to the work queue entry.

16. The computer program product of claim 15, further comprising:

instructions for performing a comparison of a queue head index and a queue tail index incremented by one to determine whether work queue is full; and

instructions for writing the work queue entry if the work queue is not full.

17. The computer program product of claim 15, wherein the work queue entry pointer comprises a real address.

18. The computer program product of claim 15, wherein the work queue entry pointer comprises a virtual address.

19. The computer program product of claim 15, wherein the instructions for writing a work queue entry pointer comprise instructions for writing the work queue entry pointer at a location referenced by a queue tail index.

20. A computer program product, in a computer readable medium, for managing a work request, comprising:

instructions for determining a location of a work queue entry pointer;

instructions for reading a work queue entry referenced by the work queue entry pointer; and

instructions for transmitting a message corresponding to the work queue entry.

21. The computer program product of claim 20, wherein the work queue entry pointer comprises a real address.

22. The computer program product of claim 20, wherein the work queue entry pointer comprises a virtual address.

23. The computer program product of claim 20, further comprising:

instructions for writing a completion queue entry corresponding to the work queue entry in memory.

24. The computer program product of claim 23, further comprising instructions for writing a completion queue entry pointer that points to the completion queue entry in a completion queue.

25. The computer program product of claim 24, wherein the instructions for writing a completion queue entry pointer comprise instructions for writing the completion queue entry pointer at a location referenced by a queue tail index.

26. A computer program product, in a computer readable medium, for managing a request for work completion information, comprising:

instructions for reading a completion queue entry pointer;

instructions for reading a completion queue entry referenced by the completion queue entry pointer; and

instructions for using the completion queue entry to return work completion information.

27. The computer program product of claim 26, wherein the completion queue entry pointer comprises a real address.

28. The computer program product of claim 26, wherein the completion queue entry pointer comprises a virtual address.

29. An apparatus for managing a work request, comprising:

a channel interface, wherein the channel interface builds a work queue entry for a work queue, writes the work queue entry in memory, and writes in the work queue a work queue entry pointer that points to the work queue entry; and

a host channel adapter coupled to the channel interface, wherein the host channel adapter processes the work queue entry.

30. An apparatus for managing a work request, comprising:

a channel interface, wherein the channel interface includes a work queue; and

a host channel adapter coupled to the channel interface, wherein the host channel adapter determines a location of a work queue entry pointer, reads a work queue entry referenced by the work queue entry pointer, and transmits a message corresponding to the work queue entry.

31. The apparatus of claim 30, wherein the host channel adapter writes a completion queue entry corresponding to the work queue entry in memory.

32. The apparatus of claim 31, wherein the channel interface includes a completion queue and wherein the host channel adapter writes a completion queue entry pointer that points to the completion queue entry in the completion queue.

33. An apparatus for managing a request for work completion information, comprising:

a channel interface, wherein the channel interface includes a completion queue; and

a host channel adapter coupled to the channel interface, wherein the host channel adapter reads a completion queue entry pointer, reads a completion queue entry referenced by the completion queue entry pointer, and uses the completion queue entry to return work completion information.