System and method for transferring data over a remote direct memory access (RDMA) network

• US 8,700,724 B2
• Filed: 08/19/2003
• Issued: 04/15/2014
• Est. Priority Date: 08/19/2002
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: McDaniel, Scott S., Elzur, Uri
• Primary Examiner(s): Nguyen, Phuoc
• Assistant Examiner(s): Belani, Kishin G
• Application Number:
  US10/643,331
• Publication Number:
  US 20040073622A1
• Time in Patent Office: 3,892 Days
• Field of Search: 709/212
• US Class Current: 709/212
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 15/167 using a common memory, e.g. mailbox
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods that provide one-shot remote direct memory access (RDMA) are provided. In one embodiment, a system that transfers data over an RDMA network may include, for example, a host. The host may include, for example, a driver and a network interface card (NIC), the driver being coupled to the NIC. The driver and the NIC may perform a one-shot initiation process and/or a one-shot completion process of an RDMA operation.

Claims

1. A system for transferring data over a remote direct memory access (RDMA) network, comprising:

a host comprising a driver and a network interface card (NIC), the driver being coupled to the NIC,

wherein a one-shot initiation process of an RDMA operation is performed between the driver and the NIC of the host, the one-shot initiation process comprising communicating a single command message comprising;

buffer command information, and

a write command to write a send command.

2. The system according to claim 1, wherein the driver posts the single command message to perform the one-shot initiation process.

3. The system according to claim 1, wherein the buffer command information comprises a command to describe pinned-down memory buffers of the host.

4. The system according to claim 3, wherein the buffer command information comprises a command to bind a portion of the pinned-down memory buffers of the host to a steering tag (STag).

5. The system according to claim 1, wherein the send command is an RDMA send message.

6. The system according to claim 4, wherein the NIC places the STag value in an optional field in a direct data placement DDP or RDMA header.

7. The system according to claim 6, wherein the NIC encodes a value into a field in the DDP or RDMA header indicating that the STag value in the optional field is valid.

8. The system according to claim 6, wherein the NIC sets one or more bits in a field in the DDP or RDMA header indicating that the STag value in the optional field is valid.

9. The system according to claim 6, wherein the NIC sets one or more bits or encodes a value into a second field in the DDP or RDMA header to advertise the portion of the pinned memory buffers of the host associated with the STag.

10. The system according to claim 1, wherein the buffer command information provides a description of a section of memory.

11. The system according to claim 1, wherein the single command message is posted to a command ring of the host.

12. The system according to claim 11, wherein the driver allocates an STag value.

13. The system according to claim 12, wherein the STag value is returned synchronously from a command call.

14. The system according to claim 12, wherein the STag value is saved in a driver command table of the host.

15. The system according to claim 14, wherein the STag value saved in a driver command table is associated with an application reference number.

16. The system according to claim 1, wherein the NIC comprises an RDMA-enabled NIC.

17. A system for transferring data over a remote direct memory access (RDMA) network, comprising:

a host comprising a driver and a network interface card (NIC), the driver being coupled to the NIC,

wherein a one-shot completion process of an RDMA operation is performed between the driver and the NIC of the host, the one-shot completion process comprising communicating a single completion message comprising;

a send complete indication, and

buffer freeing status information.

18. The system according to claim 17, wherein the NIC receives a message comprising an optional field carrying a STag value, the STag value being associated with pinned memory in a remote host.

19. The system according to claim 18, wherein a header of the message indicates the validity of the optional field with a bit flag or specified value in an encoded field.

20. The system according to claim 18, wherein the NIC de-associates the STag value with the pinned memory in the host, thereby preventing further access to the pinned memory using the de-associated STag value.

21. The system according to claim 18,

wherein the single completion message comprises the optional field carrying the STag value received by the NIC;

wherein the NIC delivers the single completion message to the driver, and

wherein the driver compares the STag value received with a STag value previously sent.

22. The system according to claim 18, wherein the NIC de-associates the STag value with previously associated SGL information.

23. The system according to claim 20, wherein the NIC frees any resources dedicated to information regarding the pinned memory.

24. A method for transferring data over an RDMA network, comprising:

initiating an RDMA write operation using a one-shot initiation process between a driver and a NIC of a host, wherein the one-shot initiation process comprises communicating a single command message comprising;

buffer command information comprising commands to insert and validate an STag value, and

a write command to write an RDMA send message;

inserting the STag value in a first field of a DDP or RDMA header of the RDMA send message; and

validating the STag value in the first field with a bit flag or other specified value in a second field of the DDP or RDMA header.

25. A method for transferring data over an RDMA network, comprising:

completing an RDMA write operation using a one-shot completion process between a NIC and a driver of a host, wherein the one-shot completion process comprises communicating a single completion message comprising;

a send complete indication,

buffer freeing status information, and

an STag value;

receiving the single completion message;

identifying the STag value in a first field of a header of the single completion message; and

validating the STag value in the first field of the header by identifying a bit flag or other specified value in a second field of the header.