Transporting operations of arbitrary size over remote direct memory access

• US 10,284,626 B2
• Filed: 04/08/2016
• Issued: 05/07/2019
• Est. Priority Date: 06/29/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Talpey, Thomas M., Kramer, Gregory Robert, Kruse, David Matthew
• Primary Examiner(s): Jean, Frantz B
• Application Number:
  US15/094,753
• Publication Number:
  US 20160226951A1
• Time in Patent Office: 1,124 Days
• Field of Search: 709212
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 1/1835 Buffer management
• CPC Class Codes: H04L 47/39 Credit based
• CPC Class Codes: H04L 47/722 at the destination endpoint, e.g. reservation of terminal resources or buffer space
• CPC Class Codes: H04L 49/9057 Arrangements for supporting packet reassembly or resequencing
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The embodiments described herein generally relate to a protocol for implementing data operations, e.g., a version of SMB, atop RDMA transports. In embodiments, systems and methods use the protocol definition, which specifies new messages for negotiating an RDMA connection and for transferring SMB2 data using the negotiated communication. A new protocol message may include new header information to determine message size, number of messages, and other information for sending the SMB2 data over RDMA. The header information is used to accommodate differences in message size requirements between RDMA and SMB2. The SMB Direct protocol allows SMB2 data to be fragmented into multiple individual RDMA messages that a receiver may then logically concatenate into a single SMB2 request or SMB2 response. The SMB Direct protocol also may allow SMB2 to transfer application data via efficient RDMA direct placement and to signal the application data'"'"'s availability when the transfer is complete.

Claims

1. A computer-implemented method for establishing a connection for exchanging data, the method comprising:

receiving, by a receiver, a first packet, wherein the packet comprises a first field and data, wherein the first field is equal to a number of bytes of a fragmented message that the receiver has yet to receive;

determining if a value stored the first field zero;

when the value is zero;

processing the data in the first packet;

when the value is not zero;

allocating a reassembly buffer to the connection;

copying the data from the first packet into the reassembly buffer;

receiving a second packet comprising second data; and

copying the second data from the second packet into the reassembly buffer.

2. The computer-implemented method of claim 1, wherein the first packet comprises one or more fields from the group consisting of:

a credits requested field;

a credits granted field;

a data offset field; and

a data length field.

3. The computer-implemented method of claim 2, wherein the credits granted field delineates how many credits were provided to a sender of the first packet.

4. The computer-implemented method of claim 1, wherein a credit is a value identifying the number of messages that may be sent by the sender.

5. The computer-implemented method of claim 1, further comprising:

determining if the second packet is a last packet;

if the second packet is the last packet, processing data in the reassembly buffer;

if the second packet is not the last packet, receiving at least a third packet and copying third data from the third packet into the reassembly buffer.

6. The computer-implemented method of claim 5, wherein the reassembly buffer includes data from at least two data packets.

7. The computer-implemented method of claim 5, wherein the determining if the first packet is a last packet comprises determining if the value stored in the first field equals zero.

8. The computer-implemented method of claim 1, further comprising:

pre-posting a receive of a negotiation request;

setting a negotiation request expiration timer;

determining if the negotiation request was received before the negotiation request expiration timer expired;

if the negotiation request was not received before the negotiation request expiration timer expired, cancelling the receive of the negotiation request;

if the negotiation request was received before the negotiation request expiration timer expired, receiving the negotiation request, wherein the negotiation request requests to establish the connection to send data using remote direct memory access;

validating the negotiation request;

processing the negotiation request;

in response to processing the negotiation request, generating a response, wherein the response allows the connection and assigns at least one credit to a sender of the negotiation request to send a packet;

sending the response.

9. The computer-implemented method of claim 8, wherein the negotiation request comprises one or more fields from the group consisting of:

a request version field;

a first credit request field;

a preferred size field indicating a first size; and

a max size field indicated a second size.

10. The computer-implemented method of claim 9, wherein the response comprises one or more fields from the group consisting of:

a response version field;

a preferred version field;

a second credit request field;

a credit granted field; and

a status field.

11. A computer storage device storing computer-executable instructions that when executed by a processor perform a method for exchanging data, the method comprising:

establishing a connection with a server;

negotiating a connection with the server, wherein the negotiation establishes a maximum number of bytes the server will receive in the connection;

determining a number of packets to send data associated with the connection to the server;

determining if fragmentation of the data is to be used;

when fragmentation of the data is not to be used;

sending a first packet to the server with the data;

when fragmentation of the data is to be used;

initializing first bytes within a transmission buffer to send as data in the first packet;

setting a first field corresponding to data length and a second field corresponding to remaining data in the first packet, wherein the second field is equal to a number of bytes of a fragmented message that the server has yet to receive;

sending the first packet to the server;

initializing bytes within the transmission buffer to send as data in the second protocol packet;

setting a third field corresponding to remaining data in the second packet; and

sending the second packet to the server.

12. The computer storage device of claim 11, wherein the determining if fragmentation is to be used comprises determining if the number of packets needed to send the data is greater than one.

13. The computer storage device of claim 12, wherein determining a number of packets to send comprises determining if a maximum number of bytes the server can receive in the connection is less than a number of bytes of data to be sent.

14. The computer storage device of claim 11, wherein negotiating the connection with the server comprises:

pre-posting a receive;

constructing a negotiation request;

sending the negotiation request to the server;

setting a negotiation expiration timer;

determining if a response to the negotiation request is received before the negotiation expiration timer expires;

if a response to the negotiation request is not received before the negotiate expiration timer expires, resending the negotiation request;

if a response to the negotiation request is received before the negotiation expiration timer expires, receiving a negotiation response;

validating the negotiation response; and

processing the negotiation response, wherein the negotiation response provides at least one credit to send the protocol packet.

15. The computer storage device of claim 14, further comprising:

if enough credits exist for sending the second packet, sending the second packet;

if enough credits do not exist to send the second packet, requesting at least one more credit from the server;

determining if enough credits are received;

if enough credits are not received, terminating the connection; and

if enough credits are received, sending the second packet.

16. A system configured to establish a connection for exchanging data, the system comprising:

at least one processor; and

memory storing computer executable instructions that, when executed by the at least one processor, cause the processor to;

receive, by a receiver, a first packet, wherein the packet comprises a first field and data, wherein the first field is equal to a number of bytes of a fragmented message that the receiver has yet to receive;

determine if a value stored by the first field is zero;

when the value stored by the first field is zero;

process the data in the first packet;

when the value stored by the first field is not zero;

allocate a reassembly buffer to the connection;

copy the data from the first packet into the reassembly buffer;

receive a second packet comprising second data; and

copy the second data from the second packet into the reassembly buffer.

17. The system of claim 16, further comprising computer executable instructions stored in the memory that, when executed by the at least one processor, cause the processor to:

determine if the second packet is a last packet;

if the second packet is the last packet, process data in the reassembly buffer; and

if the second packet is not the last packet, receive at least a third packet and copying third data from the third packet into the reassembly buffer.

18. The system of claim 17, wherein the reassembly buffer includes data from at least two data packets.

19. The system of claim 16, further comprising computer executable instructions stored in the memory that, when executed by the at least one processor, cause the processor to:

pre-post a receive of a negotiation request;

set a negotiation request expiration timer;

determine if the negotiation request was received before the negotiation request expiration timer expired;

if the negotiation request was not received before the negotiation request expiration timer expired, cancel the receive of the negotiation request;

if the negotiation request was received before the negotiation request expiration timer expired, receive the negotiation request, wherein the negotiation request requests to establish the connection to send data using remote direct memory access;

validate the negotiation request;

process the negotiation request;

in response to processing the negotiation request, generate a response, wherein the response allows the connection and assigns at least one credit to a sender of the negotiation request to send a packet; and

send the response.

20. The system of claim 16, wherein the first packet comprises one or more fields from the group consisting of:

a credits requested field;

a credits granted field;

a data offset field; and

a data length field.