Host-fabric adapter having work queue entry (WQE) ring hardware assist (HWA) mechanism

• US 6,948,004 B2
• Filed: 03/28/2001
• Issued: 09/20/2005
• Est. Priority Date: 03/28/2001
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Gasbarro, Dominic J., Leitner, Brian M.
• Primary Examiner(s): Gaffin, Jeffrey
• Assistant Examiner(s): FAROOQ, MOHAMMAD O
• Application Number:
  US09/818,902
• Publication Number:
  US 20020141424A1
• Time in Patent Office: 1,637 Days
• Field of Search: 709/218, 709/225, 709/226, 709/232, 709/238, 709/249, 709/250
• US Class Current: 709/250
• CPC Class Codes: H04L 49/35 Switches specially adapted for specific applications
• CPC Class Codes: H04L 49/358 Infiniband Switches
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 49/901 using storage descriptor, e.g. read or write pointers

Abstract

A host system is provided with one or more host-fabric adapters installed therein for connecting to a switched fabric of a data network. The host-fabric adapter may comprise at least one Micro-Engine arranged to establish connections and support data transfer operations, via a switched fabric, in response to work requests that cause instructions in a form of work queue elements “WQES” posted from a host system for said data transfer operations; and a work queue element “WQE” hardware assist “HWA” mechanism arranged to determine the starting address of each work queue element “WQE” based on queue pair (QP) context information needed for said Micro-Engine (ME) to process work requests for said data transfer operations.

Claims

1. A host-fabric adapter, comprising:

at least one Micro-Engine arranged to establish connections and support data transfer operations, via a switched fabric, in response to work requests that cause instructions in a form of work queue elements “

WQEs”

posted from a host system for said data transfer operations;

a work queue element “

WQE”

hardware assist “

HWA”

mechanism arranged to determine the starting address of each work queue element “

WQE”

based on queue pair (QP) context information needed for said Micro-Engine (ME) to process work requests for said data transfer operations;

a transport engine which contains a plurality of work queue pairs (WQPs) in which work requests in a form of work queue elements “

WQEs”

are posted to describe data transfer operations and locations of data to be moved for processing and/or transportation via said switched fabric;

wherein said work queue pairs (WOPs) each comprises;

a Send Queue (SQ) utilized as an “

initiator”

which requests normal message sends to remote Virtual Interfaces (VIs) of a remote system, remote direct memory access (RDMA) reads which request messages to be read from specific memory locations of said remote system, via said switched fabric, and remote direct memory access (RDMA) writes which request messages to be written onto specific memory locations of said remote system, via said switched fabric; and

a Receive Queue utilized as a “

responder”

which receives requests for messages from normal sends, RDMA reads and RDMA writes from said remote system, via said switched fabric; and

wherein said Micro-Engine and said work queue element “

WQE”

hardware assist “

HWA”

mechanism are configured in accordance with the “

InfiniBand™

Specification”

, and implemented as part of an Application Specific Integrated Circuit (ASIC).

2. The host-fabric adapter as claimed in claim 1, further comprising:

a local bus interface arranged to interface a local bus responsible for supporting system accessible context connections and channel operations, and turning signal data into appropriate forms for said Micro-Engine (ME) to process work requests for said data transfer operations; and

a QP context memory arranged to store QP context information needed for said Micro-Engine (ME) to process work requests for data transfer operations.

3. A host-fabric adapter, comprising:

at least one Micro-Engine arranged to establish connections and support data transfer operations, via a switched fabric, in response to work requests that cause instructions in a form of work queue elements “

WQEs”

posted from a host system for said data transfer operations;

a work queue element “

WQE”

hardware assist “

HWA”

mechanism arranged to determine the starting address of each work queue element “

WQE”

based on queue pair (QP) context information needed for said Micro-Engine (ME) to process work requests for said data transfer operations;

a local bus interface arranged to interface a local bus responsible for supporting system accessible context connections and channel operations, and turning signal data into appropriate forms for said Micro-Engine (ME) to process work requests for said data transfer operations; and

a QP context memory arranged to store QP context information needed for said Micro-Engine (ME) to process work requests for data transfer operations;

wherein said work queue element “

WQE”

hardware assist “

HWA”

mechanism is incorporated in one of said local bus interface and said QP context memory to determine the starting address of each work queue element “

WQE”

in the WQE ring, and said “

WQEs”

are programmed by host software with different given sizes including multiple Data Segments which are scatter gather lists pointing to memory regions of host system memory where message data is to be transmitted from or where incoming message data is to be written.

4. The host-fabric adapter as claimed in claim 3, further comprising a transport engine which contains a plurality of work queue pairs (WQPs) in which work requests in a form of work queue elements “

WQEs”

are posted to describe data transfer operations and locations of data to be moved for processing and/or transportation via said switched fabric.

5. The host-fabric adapter as claimed in claim 4, wherein said work queue pairs (WQPs) each comprises:

a Send Queue (SQ) utilized as an “

initiator”

which requests normal message sends to remote Virtual Interfaces (VIs) of a remote system, remote direct memory access (RDMA) reads which request messages to be read from specific memory locations of said remote system, via said switched fabric, and remote direct memory access (RDMA) writes which request messages to be written onto specific memory locations of said remote system, via said switched fabric; and

a Receive Queue utilized as a “

responder”

which receives requests for messages from normal sends, RDMA reads and RDMA writes from said remote system, via said switched fabric.

6. The host-fabric adapter as claimed in claim 3, wherein said work queue element “

WQE”

hardware assist “

HWA”

mechanism comprises;

a flip-flop arranged to receive programmed WQE size from QP context information;

a decoder arranged to decode the WQE size and supply a decoded size value based on the programmed WQE size;

a mask generator arranged to generate a WQE mask value based on the programmed WQE size; and

a multiplexer (MUX) arranged to provide one of the decoded WQE size value and the WQE mask value for each work queue (WQ) to said Micro-Engine (ME) for determining the starting address of a WQE posted on the work queue (WQ) regardless of the number of Data Segments contained in the WQE.

7. The host-fabric adapter as claimed in claim 6, wherein said mask generator includes a WQE Mask Register used to determine the starting address of the currently completed WQE in a WQE ring.

8. The host-fabric adapter as claimed in claim 6, wherein said decoder includes a WQE Size Register used to determine the starting address of the next WQE in the WQE ring.

9. The host-fabric adapter as claimed in claim 3, wherein said work queue element “

WQE”

hardware assist “

HWA”

mechanism comprises;

a WQE Size Register arranged to decode programmed WQE size from QP context information and supply a decoded WQE size value; and

a WQE Mask Register arranged to decode the programmed WQE size and supply a WQE mask value,

wherein said Micro-Engine (ME) receives the decoded WQE size value and the WQE mask value for each work queue (WQ) and executes logical AND and logical ADD functions to determine the starting address of a WQE posted on the work queue (WQ) regardless of the number of Data Segments contained in the WQE.

10. The host-fabric adapter as claimed in claim 9, wherein said Micro-Engine (ME) determines the starting address of a WQE by:

executing a logical AND function of a current WQE Offset and the WQE mask value to obtain a resultant WQE Offset indicating the starting address of the currently completed; and

executing a logical ADD function of the resultant WQE Offset and the WQE size value to obtain a resultant WQE Offset indicating the starting address of the next WQE.

11. A host-fabric adapter installed at a host system for connecting to a switched fabric of a data network, comprising:

at least one Micro-Engine (ME) arranged to establish connections and support data transfers via said switched fabric;

a serial interface arranged to receive and transmit data packets from said switched fabric for data transfer operations;

a host interface arranged to receive and transmit work requests that cause instructions in a form of work queue elements “

WQEs”

posted from said host system for data transfer operations;

a local bus interface arranged to interface a local bus responsible for supporting system accessible context connections and channel operations, and turning signal data into appropriate forms for said Micro-Engine (ME) to process work requests for said data transfer operations;

a QP context memory arranged to store QP context information needed for said Micro-Engine (ME) to process work requests for data transfer operations; and

a work queue element “

WQE”

hardware assist “

HWA”

mechanism arranged to determine the starting address of each work queue element “

WQE”

based on queue pair (QP) context information needed for said Micro-Engine (ME) to process work requests for said data transfer operations.

12. The host-fabric adapter as claimed in claim 11, wherein said work queue element “

WQE”

hardware assist “

HWA”

mechanism is incorporated in one of said local bus interface and said QP context memory to determine the starting address of each work queue element “

WQE”

in the WQE ring, and said “

WQEs”

are programmed by host software with different given sizes including multiple Data Segments which are scatter gather lists pointing to memory regions of host system memory where message data is to be transmitted from or where incoming message data is to be written.

13. The host-fabric adapter as claimed in claim 12, wherein said work queue element “

WQE”

hardware assist “

HWA”

mechanism comprises;

a flip-flop arranged to receive programmed WQE size from QP context information;

a decoder arranged to decode the WQE size and supply a decoded size value based on the programmed WQE size;

a mask generator arranged to generate a WQE mask value based on the programmed WQE size; and

a multiplexer (MUX) arranged to provide one of the decoded WQE size value and the WQE mask value for each work queue (WQ) to said Micro-Engine (ME) for determining the starting address of a WQE posted on the work queue (WQ) regardless of the number of Data Segments contained in the WQE.

14. The host-fabric adapter as claimed in claim 13, wherein said mask generator includes a WQE Mask Register used to determine the starting address of the currently completed WQE in a WQE ring.

15. The host-fabric adapter as claimed in claim 13, wherein said decoder includes a WQE Size Register used to determine the starting address of the next WQE in the WQE ring.

16. The host-fabric adapter as claimed in claim 12, wherein said work queue element “

WQE”

hardware assist “

HWA”

mechanism comprises;

a WQE Size Register arranged to decode programmed WQE size from QP context information and supply a decoded WQE size value; and

a WQE Mask Register arranged to decode the programmed WQE size and supply a WQE mask value,

wherein said Micro-Engine (ME) receives the decoded WQE size value and the WQE mask value for each work queue (WQ) and executes logical AND and ADD functions to determine the starting address of a WQE posted on the work queue (WQ) regardless of the number of Data Segments contained in the WQE.

17. The host-fabric adapter as claimed in claim 16, wherein said Micro-Engine (ME) determines the starting address of a WQE by:

executing a logical AND function of a current WQE Offset and the WQE mask value to obtain a resultant WQE Offset indicating the starting address of the currently completed; and

executing a logical ADD function of the resultant WQE Offset and the WQE size value to obtain a resultant WQE Offset indicating the starting address of the next WQE.

18. A method of determining the starting address of a work queue element “

WQE”

in a WQE ring for-processing outstanding data transfer operations in a host-fabric adapter installed at a host system, comprising;

generating a WQE size value based on a programmed WQE size from context information;

generating a WQE mask value based on the programmed WQE size from the context information;

executing a logical AND function of a current WQE Offset obtained from the context information and the WQE mask value to supply a resultant WQE Offset indicating the starting address of the currently completed; and

executing a logical ADD function of the resultant WQE Offset and the WQE size value to supply a resultant WQE Offset indicating the starting address of the next WQE.

19. A computer readable medium having computer-executable instructions for performing a method of determining the starting address of a work queue element “

WQE”

in a WQE ring so as to process outstanding data transfer operations in a host-fabric adapter installed at a host system, said method comprising;

generating a WQE size value based on a programmed WQE size from context information;

generating a WQE mask value based on the programmed WQE size from the context information;

executing a logical AND function of a current WQE Offset obtained from the context information and the WQE mask value to supply a resultant WQE Offset indicating the starting address of the currently completed; and

executing a logical ADD function of the resultant WQE Offset and the WQE size value to supply a resultant WQE Offset indicating the starting address of the next WQE.