Intelligent network adaptor with end-to-end flow control

• US 8,356,112 B1
• Filed: 09/29/2011
• Issued: 01/15/2013
• Est. Priority Date: 05/11/2007
• Status: Active Grant

• Current Assignee: Chelsio Communications Incorporated
• Original Assignee: Chelsio Communications Incorporated
• Inventors: Michailidis, Dimitrios, Noureddine, Wael, Marti, Felix A., Eiriksson, Asgeir Thor
• Primary Examiner(s): DENNISON, JERRY B
• Application Number:
  US13/249,077
• Time in Patent Office: 474 Days
• Field of Search: None
• US Class Current: 709/234
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/27 Evaluation or update of window size, e.g. using information derived from acknowledged [ACK] packets
• CPC Class Codes: H04L 47/30 in combination with information about buffer occupancy at either end or at transit nodes
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures

Abstract

A host is coupled to a network via an intelligent network adaptor. The host is executing an application configured to receive application data from a peer via the network and the intelligent network adaptor using a stateful connection according to a connection-oriented protocol. The intelligent network adaptor performs protocol processing of the connection. Application data is copied from host memory not configured for access by the application (possibly OS-associated host memory) to host memory associated with the application (application-associated host memory). The application data is received from the peer by the intelligent network adaptor and copied to host memory not configured for access by the application. The operating system selectively provides, to the intelligent network adaptor, information of the memory associated with the application. At least one portion of the application data for the connection is provided directly from the intelligent network adaptor to the memory associated with the application.

Claims

1. A method of operating a stateful connection between a host and a peer via a network, the host executing an application configured to receive data packets, including application data, from the peer via the network using the stateful connection between the host and the peer according to a connection-oriented protocol, the method comprising:

performing protocol processing for the connection with the peer to, at least in part, obtain the application data from the data packets received from the peer, including indicating a receive window to the peer;

placing the obtained application data to host memory application buffer associated with the application, the host memory application buffer not being associated with a host operating system;

causing the indicated receive window to be increased based on the host memory application buffer becoming available due to the application consuming application data from the application buffer; and

providing flow control by generating the receive window indication, including maintaining an indication of receive window credits based on an amount of the application data placed to the host memory application buffer and further based on the application data in the host memory application buffer being consumed by the application.

2. The method of claim 1, further comprising:

causing the indicated receive window to be reduced based on the application data being placed in the host memory application buffer by an intelligent network adaptor that couples the host to a peer a network to operate the stateful connection between the host and the peer via the network.

3. A method of operating an intelligent network adaptor that couples a host to a network, the host executing an application configured to receive data packets, including application data from a peer via the network and the intelligent network adaptor using a stateful connection between the host and the peer according to a connection-oriented protocol, the method comprising:

by the intelligent network adaptor, receiving, from the host, a plurality of indications that the application has consumed application data from a host memory application buffer associated with the application, the host memory application buffer not being associated with a host operating system; and

by the intelligent network adaptor, generating a receive window indication based at least in part on the received indications of consumption of the application data from the host memory application buffer;

the host memory application buffer is larger than an amount of memory allowed by the connection-oriented protocol for flow control; and

the host is configured to expose, for use by protocol processing of the connection at any particular time, a portion of the host memory application buffer that is no larger than the amount of memory allowed by the connection-oriented protocol for flow control

the host maintains the host memory application buffer as a plurality of overlapping windows into the host memory application buffer, each window no larger than the amount of memory allowed by the connection-oriented protocol for flow control; and

the host selectively exposes to the protocol processing one of the windows at any particular time.

4. The method of claim 3, wherein:

the host maintains the host memory application buffer as a ring structure.

5. The method of claim 3, wherein:

maintaining the host memory application buffer as a plurality of overlapping windows includes maintaining the host memory application buffer as a ring structure, including maintaining producer and consumer pointers into a currently-exposed window, including advancing the producer pointer based on in-order application data being placed in the host memory application buffer and advancing the consumer pointer based on application data being consumed by the application from the host memory application buffer.

6. The method of claim 5, wherein:

maintaining the host memory application buffer as a plurality of overlapping windows includes, as the consumer pointer is advanced based on consumption of application data by the application, releasing credits corresponding to the connection from the host to the intelligent network adaptor; and

the intelligent network adaptor generates the receive window for the connection based at least in part on the released credits.

7. The method of claim 6, wherein:

releasing credits corresponding to the connection from the host to the intelligent network adaptor includes accumulating credits corresponding to the connection and bunching the release of accumulated credits from the host to the intelligent network adaptor.

8. The method of claim 5, wherein:

advancing the producer pointer based on in-order application data being placed in the host memory application buffer includes the intelligent network adaptor accumulating notifications to the host of application data being placed in the host memory application buffer to moderate the notifications being provided to the host.

9. The method of claim 3, wherein:

maintaining the producer and consumer pointers into the currently-exposed window includes constraining the producer pointer from being advanced only as far as the one-window'"'"'s worth of memory away from the consumer pointer.

10. The method of claim 5, wherein:

maintaining the host memory application buffer as a plurality of overlapping windows includes, as the producer pointer is advanced based on placing of application data by the intelligent network adaptor, decrementing receive windows credits corresponding to the connection; and

the intelligent network adaptor generates the receive window based at least in part on receive window credits corresponding to the connection.

11. An intelligent network adaptor that couples a host to a network, the host executing an application configured to receive data packets, including application data, from a peer via the network and the intelligent network adaptor using a stateful connection between the host and the peer according to a connection-oriented protocol, wherein the intelligent network adaptor performs protocol processing of the connection, the intelligent network adaptor comprising hardware configured to:

perform protocol processing for the connection with the peer to, at least in part, obtain the application data from the data packets received from the peer, including indicating a receive window to the peer;

place application data, received from the peer via the stateful connection, from the intelligent network adaptor directly to host memory application buffer associated with the application without the application data being first provided from the intelligent network adaptor to host memory buffer associated with a host operating system and not specifically with the application;

cause the indicated receive window to be increased based on the application buffer becoming available due to the application consuming application data from the application buffer; and

provide flow control by generating the receive window indication, including maintaining an indication of receive window credits based on an amount of the application data placed from the intelligent network adaptor to host memory application buffer and further based on the application data in the host memory application buffer being consumed by the application.

12. The intelligent network adaptor of claim 11, further configured to:

cause the indicated receive window to be reduced based on the application data being placed in the host memory application buffer by the intelligent network adaptor.

13. The intelligent network adaptor of claim 11, further configured to:

place application data, received from the peer via the stateful connection, from the intelligent network adaptor to the host memory buffer associated with the operating system and not specifically with the application and, based thereon, causing the indicated receive window to be reduced.

14. An intelligent network adaptor that couples a host to a network, the host executing an application configured to receive data packets, including application data from a peer via the network and the intelligent network adaptor using a stateful connection between the host and the peer according to a connection-oriented protocol, wherein the intelligent network adaptor performs protocol processing of the connection, the intelligent network adaptor comprising hardware configured to:

perform protocol processing for the connection with the peer to, at least in part, obtain the application data from the data packets received from the peer, including providing a receive window to the peer

place application data, received from the peer via the stateful connection, from the intelligent network adaptor directly to host memory application buffer associated with the application without the application data being first provided from the intelligent network adaptor to host memory buffer associated with a host operating system and not specifically with the application; and

receive, from the host, an indication that the application has consumed application data from the host memory application buffer associated with the application; and

generate the receive window based at least in part on the received indications of consumption of the application data from the host memory application buffer, wherein

the host memory application buffer is larger than an amount of memory allowed by the connection-oriented protocol for flow control;

the host is configured to expose to the interface adaptor, for use with the connection at any particular time, a portion of the host memory application buffer that is no larger than the amount of memory allowed by the connection-oriented protocol for flow control;

the host maintains the host memory application buffer as a plurality of overlapping windows into the host memory application buffer, each window no larger than the amount of memory allowed by the connection-oriented protocol for flow control; and

the host selectively exposes to the protocol processing one of the windows at any particular time.