Protocol-independent support of remote DMA

• US 6,675,200 B1
• Filed: 05/10/2000
• Issued: 01/06/2004
• Est. Priority Date: 05/10/2000
• Status: Expired due to Term

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Sapuntzakis, Constantine P., Cheriton, David R.
• Primary Examiner(s): Lim, Krisna
• Application Number:
  US09/569,850
• Time in Patent Office: 1,336 Days
• Field of Search: 370/464, 370/471, 370/472, 370/476, 710/22, 709/232, 709/212
• US Class Current: 709/212
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/12 Protocol engines
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/161 Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A remote DMA (RDMA) shim protocol laid atop an existing network data transfer protocol but logically underneath higher level disk and file access protocols specifies the portion of a data packet to be transferred directly into a separate area of memory such as an application layer memory buffer. This RDMA protocol identifies the area of memory into which the data should be delivered, a data ID, data start, data length or end, and flag bits. Option fields added to (or already present in) the transport protocol describe the RDMA transfer. On reception of a packet specifying RDMA, the receiving device delivers the data directly into the correct memory location using the RDMA description data. In some embodiments of the present invention, the RDMA shim protocol is implemented with TCP options specifically introduced to enable RDMA and thus reduce the overhead of transferring and receiving data with a TCP-based protocol such as NFS or HTTP. Use of a TCP option technique enables the construction of simple hardware accelerators to copy data directly from the incoming packet into application memory buffers, thus avoiding expensive copies within the protocol stack. Alternatively, software techniques may be used to perform direct copying to the application memory buffer. The RDMA protocol options thus enable the protocol stack to decrease the number of copies required to perform data transfers, thus lowering overhead costs.

Claims

1. A method of facilitating remote direct memory access (RDMA) in a data communications system comprising the steps of:

receiving a packet according to an intermediate protocol layer definition, wherein said intermediate protocol layer definition provides for a plurality of RDMA byte codes in the packet header, and wherein the packet header comprises one or more RDMA byte codes; and

processing said packet based at least in part on said RDMA byte codes, said processing further comprising;

reading said byte codes to identify an RDMA data payload within said packet; and

storing said RDMA data payload in a memory without copying said RDMA payload.

2. The method of claim 1, wherein:

said byte codes comprise

an RDMA identifier;

an RDMA, data offset value; and

an RDMA data length value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory using said data offset value and said data length value.

3. The method of claim 1, wherein:

said byte codes comprise;

an RDMA identifier;

a buffer offset value;

a sequence start value; and

a sequence end value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory starting at an address designated by said buffer offset value and said sequence start value.

4. A method of facilitating remote direct memory access (RDMA) in a data communications system comprising the steps of:

receiving a packet according to an adapted transport layer protocol definition, wherein said adapted transport layer protocol definition provides for a plurality of RDMA byte codes in the packet header, and wherein the packet header comprises one or more RDMA byte codes; and

processing said packet based at least in part on said RDMA byte codes, said processing further comprising;

reading said byte codes to identify an RDMA data payload within said packet; and

storing said RDMA data payload in a memory without copying said RDMA payload.

5. The method of claim 4, wherein said adapted transport layer protocol definition is based on the Transmission Control Protocol (TCP).

6. The method of claim 4, wherein said byte codes are provided within the option bytes of said Transmission Control Protocol (TCP).

7. The method of claim 4, wherein:

said byte codes comprise

an RDMA identifier;

an RDMA data offset value; and

an RDMA data length value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory using said data offset value and said data length value.

8. The method of claim 4, wherein:

said byte codes comprise;

an RDMA identifier;

a buffer offset value;

a sequence start value; and

a sequence end value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory starting at an address designated by said buffer offset value and said sequence start value.

9. A computer system for facilitating remote direct memory access (RDMA) in a data communications system, said computer system comprising a network interface, a memory, and a processor configured to:

receive a packet according to an intermediate protocol layer definition in said network interface, wherein said intermediate protocol layer definition provides for a plurality of RDMA byte codes in the packet header, and wherein the packet header comprises one or more RDMA byte codes; and

process said packet based in said processor at least in part on said RDMA byte codes, wherein said processor is connected to said network interface,

read said byte codes to identify an RDMA data payload within said packet in said processor; and

store said RDMA data payload in said memory without copying said RDMA payload.

10. The computer system of claim 9, wherein:

said byte codes comprise

an RDMA identifier;

an RDMA data offset value; and

an RDMA data length value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory using said data offset value and said data length value.

11. The computer system of claim 9, wherein:

said byte codes comprise;

an RDMA identifier;

a buffer offset value;

a sequence start value; and

a sequence end value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory starting at an address designated by said buffer offset value and said sequence start value.

12. A computer-readable storage medium, comprising computer instructions for:

receiving a packet according to an intermediate protocol layer definition, wherein said intermediate protocol layer definition provides for a plurality of RDMA byte codes in the packet header, and wherein the packet header comprises one or more RDMA byte codes; and

processing said packet based at least in part on said RDMA byte codes, said processing further comprising;

reading said byte codes to identify an RDMA data payload within said packet; and

storing said RDMA data payload in a memory without copying said RDMA payload.

13. The computer-readable storage medium of claim 12, wherein:

said byte codes comprise

an RDMA identifier;

an RDMA data offset value; and

an RDMA data length value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory using said data offset value and said data length value.

14. The computer-readable storage medium of claim 12, wherein:

said byte codes comprise;

an RDMA identifier;

a buffer offset value;

a sequence start value; and

a sequence end value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory starting at an address designated by said buffer offset value and said sequence start value.

15. A computer data signal embodied in a carrier wave, comprising computer instructions for:

receiving a packet according to an intermediate protocol layer definition, wherein said intermediate protocol layer definition provides for a plurality of RDMA byte codes in the packet header, and wherein the packet header comprises one or more RDMA byte codes; and

processing said packet based at least in part on said RDMA byte codes, said processing further comprising;

reading said byte codes to identify an RDMA data payload within said packet; and

storing said RDMA data payload in a memory without copying said RDMA payload.

16. The computer data signal of claim 15, wherein:

said byte codes comprise

an RDMA identifier;

an RDMA data offset value; and

an RDMA data length value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory using said data offset value and said data length value.

17. The computer data signal of claim 15, wherein:

said byte codes comprise;

an RDMA identifier;

a buffer offset value;

a sequence start value; and

a sequence end value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory starting at an address designated by said buffer offset value and said sequence start value.

18. A computer system for facilitating remote direct memory access (RDMA) in a data communications system, said computer system:

means for receiving a packet according to an intermediate protocol layer definition, wherein said intermediate protocol layer definition provides for a plurality of RDMA byte codes in the packet header, and wherein the packet header comprises one or more RDMA byte codes; and

means for processing said packet based at least in part on said RDMA byte codes, said means for processing further comprising;

means for reading said byte codes to identify an RDMA data payload within said packet; and

means for storing said RDMA data payload in a memory without copying said RDMA payload.

19. The computer system of claim 18, wherein:

said byte codes comprise

an RDMA identifier;

an RDMA data offset value; and

an RDMA data length value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory using said data offset value and said data length value.

20. The computer system of claim 18, wherein:

said byte codes comprise;

an RDMA identifier;

a buffer offset value;

a sequence start value; and

a sequence end value; and

said memory is identified by said RDMA identifier; and

wherein said RDMA data is written to said memory starting at an address designated by said buffer offset value and said sequence start value.