Adaptive Low Latency Receive Queues

• US 20090077567A1
• Filed: 09/14/2007
• Published: 03/19/2009
• Est. Priority Date: 09/14/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Craddock, David, Gregg, Thomas A., Schlipf, Thomas
• Granted Patent: US 8,265,092 B2
• Time in Patent Office: Days
• US Class Current: 719/314
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence

Abstract

A receive queue provided in a computer system holds work completion information and message data together. An InfiniBand hardware adapter sends a single CQE+ message data to the computer system that includes the completion Information and data. This information is sufficient for the computer system to receive and process the data message, thereby providing a highly scalable low latency receiving mechanism.

Claims

1. An adapter for processing and sending received low latency data packets to a low latency receive queue and for sending standard data packets for standard processing, the adapter comprising:

a processing moans for receiving low latency data packets each having a packet header and payload data, for processing information contained in the packet header to construct completion queue information, for constructing a low latency data message comprising the completion queue information and the payload data, and for sending the constructed data message to a receive queue in a single transfer.

2. The adapter of claim 1, wherein the receive queue comprises:

a plurality of slots each for storing the completion queue information and the payload data.

3. The adapter of claim 2, wherein the completion queue information comprises data for identifying a length of the low latency data message.

4. The adapter of claim 3, wherein each of said plurality of slots contains data storage space equal to an amount of data in a cache line.

5. The adapter of claim 3, wherein the adapter comprises:

a first register for identifying a number of available slots in the low latency receive queue; and

a second register for identifying a number of recently released slots in the receive queue to be added to the first register, and wherein a host system updates the second register.

6. The adapter of claim 5, wherein the adapter is coupled to a low latency receive queue page table for storing memory location addresses of the low latency receive queue and wherein the adapter accesses the low latency receive queue page table for obtaining the location addresses.

7. The adapter of claim 4, wherein completion queue information stored in the low latency receive queue comprises a bit for indicating that the completion queue information and payload data of a low latency data message has been stored in the receive queue and is visible to a processor in a host system.

8. The adapter of claim 7, wherein the information identifying the length of the low latency data message is used by a host system to identify a next slot wherein a subsequent CQE valid bit will be stored for clearing the subsequent CQE valid bit before its message data is stored.

9. The adapter of claim 7, wherein a low latency data message longer than one slot is stored in the receive queue such that the completion queue information is stored in the first slot and the first slot is stored last.

10. A computer system comprising:

a memory;

a processor connected to said memory;

an adapter slot for coupling a network adapter to the computer system;

a low latency receive queue for storing low latency data messages each comprising a completion queue element and data which is received by the computer system via the adapter slot from the network adapter; and

the completion queue element comprising a valid bit indicating that the entire completion queue element and data is stored in the receive queue and is visible to the processor.

11. The computer system of claim 10, wherein the computer system further includes a page table for identifying to the network adapter system memory locations where the receive queue slots are located.

12. The computer system of claim 10, wherein the computer system sends receive queue status information to the attached network adapter for identifying a number of slots available in the receive queue.

13. The computer system of claim 12, wherein the receive queue comprises a plurality of slots each for storing one of the low latency data messages received from the attached adapter.

14. The computer system of claim 10, further comprising:

a plurality of additional adapter slots having network adapters attached therein each comprising a low latency virtual lane input buffer.

15. The computer system of claim 13, wherein each of said plurality of slots is capable of storing an amount of data equal to that stored in a cache line of the computer system.

16. A hardware adapter for coupling to a host system and to a network, the hardware adapter comprising:

a standard latency input buffer and a low latency virtual lane input buffer for storing standard and low latency network packets from the network, each low latency packet including message data and header data for the adapter to construct a single data message comprising completion queue information and data sufficient for the host system to process the data message.

17. The hardware adapter of claim 16, wherein the hardware adapter further comprises a plurality of queue pair context tables and the header data includes a queue pair number for identifying which queue pair context table is associated with an incoming packet.

18. The hardware adapter of claim 16, wherein a length of the single data message is equivalent to a length of a cache line of the host system.

19. The hardware adapter of claim 16, wherein the single data message comprises a completion queue valid bit that is set by the hardware adapter only after the single data message has been received by the host system and is visible to a processor of the host system.

20. The hardware adapter of claim 19, wherein the host system comprises a receive queue having a plurality of slots for storing a plurality of said single data messages sent by the hardware adapter.

21. The hardware adapter of claim 20, wherein the hardware adapter clears a completion queue valid bit of a succeeding data message to be stored in a next slot of the receive queue prior to sending a preceding single data message to the receive queue.

22. A program storage device readable by machine, the program storage device tangibly embodying a program of instructions executable by the machine to perform the method steps comprising:

determining if an incoming data packet is a low latency unreliable datagram;

if the incoming packet is determined to be the low latency unreliable datagram, then receiving the packet in a dedicated low latency input buffer;

reading a queue pair number in the received packet;

obtaining queue pair context information corresponding to the queue pair number; and

constructing a completion queue element based on the queue pair context information and on information in a header of the received packet and combining the completion queue element with payload data of the received packet into a current single data message sufficient for the host computer system to complete processing of the data message.

23. The program product according to claim 22, the method further comprising the steps of:

determining a size of the current single data message based on information in the header of the received packet and on the size of the completion queue element; and

providing a receive queue in the host system comprising a plurality of slots each for storing, until processed by the host system, one of a plurality of single data messages or, if any single data message is larger than one slot then storing a portion of the single data message, wherein each single data message begins at a slot boundary with the completion queue element preceding the data portion.

24. The program product according to claim 23, the method further comprising the step of:

providing a valid bit in the completion queue element for indicating to a processor of the host system, when the valid bit is set, that the current single data message has been entirely stored in the receive queue and is visible to the processor.

25. The program product according to claim 24, the method further comprising the steps of:

determining at which slot in the receive queue a single data message immediately succeeding the current single data message will begin;

clearing a valid bit in the completion queue element field of the single data message immediately succeeding the current single data message in the receive queue; and

after clearing the valid bit sending the current single data message to the host system for storing in the receive queue,