Method, system, and program for managing data transmission through a network

• US 7,870,268 B2
• Filed: 09/15/2003
• Issued: 01/11/2011
• Est. Priority Date: 09/15/2003
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Beverly, Harlan T., Tsao, Gary Y., Choubal, Ashish, Arizpe, Arturo L.
• Primary Examiner(s): Srivastava; Vivek
• Assistant Examiner(s): Goodchild; William J
• Application Number:
  US10/663,026
• Publication Number:
  US 20050060442A1
• Time in Patent Office: 2,675 Days
• Field of Search: 709/228
• US Class Current: 709/228
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/193 at the transport layer, e.g. TCP related
• CPC Class Codes: H04L 47/27 Evaluation or update of window size, e.g. using information derived from acknowledged [ACK] packets
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/163 In-band adaptation of TCP data exchange; In-band control procedures

Abstract

Provided are a method, system, and program for managing data transmission from a source to a destination through a network. The destination imposes a window value on the source which limits the quantity of data packets which can be sent from the source to the destination without receiving an acknowledgment of being received by the destination. In one embodiment, the source imposes a second window value, smaller than the destination window value, which limits even further the quantity of data packets which can be sent from the source to the destination without receiving an acknowledgment of being received by the destination. In another embodiment, a plurality of direct memory access connections are established between the source and a plurality of specified memory locations of a plurality of destinations. The source imposes a plurality of message limits, each message limit imposing a separate limit for each direct memory access connection on the quantity of messages sent from the source to the specified memory location of the direct memory access connection associated with the message limit and lacking a message acknowledgment of being received by the destination of the direct memory access connection associated with the message limit.

Claims

1. A method for sending data in a computer system, comprising:

establishing a plurality of active direct memory access connections between the host and a plurality of destinations;

establishing for each connection a Protocol Control Block, each Protocol Control Block having a first window field, a virtual window field and a message limit field, for the associated connection;

receiving from each destination a first window value representing a first quantity of data packets for the connection associated with the destination, and storing the received first window value in a first window field of the Protocol Control Block associated with the connection;

for each Protocol Control Block, storing the value of a second quantity of data packets in the virtual window field of the Protocol Control Block, wherein the second quantity of each connection is less than the first window field value of the connection and is based, at least in part, on the number of active connections of the host;

sending packets of data from said host to each destination;

receiving an acknowledgment from each destination for each packet of data received by each destination wherein the first window value of each connection represents a limit imposed on said host by the destination of the connection on the quantity of data packets sent from said host to the destination of the connection and lacking an acknowledgment of being received by the destination of the connection; and

limiting the number of packets sent by said host to each connection, but not acknowledged as received by the destination of each connection, to the value of the virtual window field of the Protocol Control Block associated with the connection, wherein the value of the virtual window field of the Protocol Control Block associated with the connection is less than the value of the first window field of the Protocol Control Block associated with the connection;

sending a plurality of messages to specified memory locations of the destinations of the direct memory access connections wherein each message comprises a plurality of data packets;

receiving message acknowledgments, each message acknowledgment being sent by a destination for each message received by the destination; and

for each connection, establishing a message limit and storing the value of the message limit in the message limit field of the Protocol Control Block associated with the connection; and

for each connection, limiting the number of messages sent by said host through the connection, but not acknowledged as received by the destination of the connection, to the value of the message limit field of the Protocol Control Block associated with the connection so that each message limit imposes a separate limit for each direct memory access connection on the quantity of messages sent from said host to the specified memory location of the direct memory access connection associated with the message limit and lacking a message acknowledgment of being received by the destination of the direct memory access connection associated with the message limit, each message limit providing a further limit on the associated connection in addition to the packet limit imposed by the value of the virtual window field of the Protocol Control Block associated with the connection.

2. The method of claim 1, wherein each direct memory access connection includes a network interface between an application of said host and a network connecting the host to the plurality of destinations and wherein said network interface includes a queue for each direct memory access connection and adapted to queue messages to be sent through the direct memory access connection associated with each queue, and wherein said each sending of a message to specified memory location of the destination of a direct memory access connection includes queuing the message in the network interface queue associated with the direct memory access connection;

and

wherein the queuing of messages in the network interface queue associated with a direct memory access connection is suspended when the quantity of messages sent from said host to the specified memory location of the associated direct memory access connection and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection reaches the separate message limit imposed on the direct memory access connection associated with the network interface queue.

3. The method of claim 2, wherein the queuing of messages in the network interface queue associated with a direct memory access connection is resumed when the quantity of messages sent from said host to the specified memory location of the associated direct memory access connection and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection is less than the separate message limit imposed on the direct memory access connection associated with the network interface queue.

4. The method of claim 3 wherein each active direct memory access connection includes a Transmission Control Protocol connection between the host and the destination and wherein said first window value is a Transmission Control Protocol send window value and

wherein each direct memory access connection is a Remote Direct Memory Access connection between the host and the destination of the direct memory access connection and each message is a Remote Direct Memory Access message.

5. The method of claim 4 wherein said network interface has a pool of empty messages which imposes a limit on the total quantity of messages sent from said host to all the specified memory locations of all the direct memory access connections and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection and wherein each message limit is less than the network interface pool of empty messages.

6. The method of claim 1 wherein each active direct memory access connection includes a Transmission Control Protocol connection between the host and the destination and wherein said first window value is a Transmission Control Protocol send window value.

7. The method of claim 6 further comprising,

in response to the destination reducing the size of the Transmission Control Protocol send window value to a third quantity less than the second quantity, limiting the number of packets sent by said host, but not acknowledged as received by said destination, to a fourth quantity of data packets no greater than the reduced size of the Transmission Control Protocol send window value;

and

storing the value of the fourth quantity of data packets in the virtual window field of the associated Protocol Control Block.

8. The method of claim 1 further comprising changing the size of a message limit of an active direct memory access connection prior to sending at least one message through the associated direct memory access connection.

9. The method of claim 8 wherein each message limit is based, at least in part, on the number of active direct memory access connections of the host.

10. The method of claim 1 wherein each message limit is based, at least in part, on the number of active direct memory access connections of the host.

11. The method of claim 1 further comprising changing the size of the second quantity of packets limiting the number of packets sent by the host but not acknowledged as received by the destination prior to sending at least one packet.

12. A system adapted to communicate with data storage and a destination having memory locations, comprising:

a system memory;

a processor coupled to the system memory;

a network adaptor;

a data storage controller for managing Input/Output (I/O) access to the data storage; and

a device driver executable by the processor in the memory, wherein at least one of the device driver and the network adaptor is adapted to;

establish a plurality of active direct memory access connections between the system and a plurality of destinations;

establish for each connection a Protocol Control Block, each Protocol Control Block having a first window field, a virtual window field and a message limit field, for the associated connection;

receive from each destination a first window value representing a first quantity of data packets for the connection associated with the destination, and store the received first window value in a first window field of the Protocol Control Block associated with the connection;

for each Protocol Control Block, store the value of a second quantity of data packets in the virtual window field of the Protocol Control Block, wherein the second quantity of each connection is less than the first window field value of the connection and is based, at least in part, on the number of active connections of the system;

send packets of data from said system to each destination;

receive an acknowledgment from each destination for each packet of data received by each destination wherein the first window value of each connection represents a limit imposed on said system by the destination of the connection on the quantity of data packets sent from said system to the destination of the connection and lacking an acknowledgment of being received by the destination of the connection; and

limit the number of packets sent by said system to each connection, but not acknowledged as received by the destination of each connection, to the value of the virtual window field of the Protocol Control Block associated with the connection, wherein the value of the virtual window field of the Protocol Control Block associated with the connection is less than the value of the first window field of the Protocol Control Block associated with the connection;

send a plurality of messages to specified memory locations of the destinations of the direct memory access connections wherein each message comprises a plurality of data packets;

receive message acknowledgments, each message acknowledgment being sent by a destination for each message received by the destination; and

for each connection, establish a message limit and store the value of the message limit in the message limit field of the Protocol Control Block associated with the connection; and

for each connection, limit the number of messages sent by said system through the connection, but not acknowledged as received by the destination of the connection, to the value of the message limit field of the Protocol Control Block associated with the connection so that each message limit imposes a separate limit for each direct memory access connection on the quantity of messages sent from said system to the specified memory location of the direct memory access connection associated with the message limit and lacking a message acknowledgment of being received by the destination of the direct memory access connection associated with the message limit, each message limit providing a further limit on the associated connection in addition to the packet limit imposed by the value of the virtual window field of the Protocol Control Block associated with the connection.

13. The system of claim 12, wherein at least one of the device driver and the network adaptor is adapted to provide a queue for each direct memory access connection and adapted to queue messages to be sent through the direct memory access connection associated with each queue, and wherein in each sending of a message to specified memory location of the destination of a direct memory access connection, at least one of the device driver and the network adaptor is adapted to queue the message in the queue associated with the direct memory access connection;

and to suspend the queuing of messages in the queue associated with a direct memory access connection when the quantity of messages sent from said system to the specified memory location of the associated direct memory access connection and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection reaches the separate message limit imposed on the direct memory access connection associated with the queue.

14. The system of claim 13, wherein at least one of the device driver and the network adaptor is adapted to resume the queuing of messages in the queue associated with a direct memory access connection when the quantity of messages sent from said system to the specified memory location of the associated direct memory access connection and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection is less than the separate message limit imposed on the direct memory access connection associated with the network interface queue.

15. The system of claim 14 wherein each active direct memory access connection includes a Transmission Control Protocol connection between the host and the destination and wherein said first window value is a Transmission Control Protocol send window value and wherein each direct memory access connection is a Remote Direct Memory Access connection between the system and the destination of the direct memory access connection and each message is a Remote Direct Memory Access message.

16. The system of claim 15 wherein at least one of the device driver and the network adaptor is adapted to provide a pool of empty messages which imposes a limit on the total quantity of messages sent from said system to all the specified memory locations of all the direct memory access connections and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection and wherein each message limit is less than the pool of empty messages.

17. The system of claim 12, wherein each active direct memory access connection includes a Transmission Control Protocol connection between the system and the destination and wherein said first window value is a Transmission Control Protocol send window value.

18. The system of claim 17 wherein at least one of the device driver and the network adaptor is adapted to:

in response to the destination reducing the size of the Transmission Control Protocol send window value to a third quantity less than the second quantity, limit the number of packets sent by said system, but not acknowledged as received by said destination, to a fourth quantity of data packets no greater than the reduced size of the Transmission Control Protocol send window value; and

store the value of the fourth quantity of data packets in the virtual window field of the associated Protocol Control Block.

19. The system of claim 12, wherein the data storage comprises a magnetic storage medium.

20. An article of manufacture comprising a computer readable storage medium having code executed by a processor for managing data transmission through a network, wherein the article of manufacture causes operations to be performed, the operations comprising:

establishing a plurality of active direct memory access connections between the host and a plurality of destinations;

establishing for each connection a Protocol Control Block, each Protocol Control Block having a first window field, a virtual window field and a message limit field, for the associated connection;

receiving from each destination a first window value representing a first quantity of data packets for the connection associated with the destination, and storing the received first window value in a first window field of the Protocol Control Block associated with the connection;

for each Protocol Control Block, storing the value of a second quantity of data packets in the virtual window field of the Protocol Control Block, wherein the second quantity of each connection is less than the first window field value of the connection and is based, at least in part, on the number of active connections of the host;

sending packets of data from said host to each destination;

receiving an acknowledgment from each destination for each packet of data received by each destination wherein the first window value of each connection represents a limit imposed on said host by the destination of the connection on the quantity of data packets sent from said host to the destination of the connection and lacking an acknowledgment of being received by the destination of the connection; and

limiting the number of packets sent by said host to each connection, but not acknowledged as received by the destination of each connection, to the value of the virtual window field of the Protocol Control Block associated with the connection, wherein the value of the virtual window field of the Protocol Control Block associated with the connection is less than the value of the first window field of the Protocol Control Block associated with the connection;

sending a plurality of messages to specified memory locations of the destinations of the direct memory access connections wherein each message comprises a plurality of data packets;

receiving message acknowledgments, each message acknowledgment being sent by a destination for each message received by the destination; and

for each connection, establishing a message limit and storing the value of the message limit in the message limit field of the Protocol Control Block associated with the connection; and

for each connection, limiting the number of messages sent by said host through the connection, but not acknowledged as received by the destination of the connection, to the value of the message limit field of the Protocol Control Block associated with the connection so that each message limit imposes a separate limit for each direct memory access connection on the quantity of messages sent from said host to the specified memory location of the direct memory access connection associated with the message limit and lacking a message acknowledgment of being received by the destination of the direct memory access connection associated with the message limit, each message limit providing a further limit on the associated connection in addition to the packet limit imposed by the value of the virtual window field of the Protocol Control Block associated with the connection.

21. The article of manufacture of claim 20, wherein each direct memory access connection includes a network interface between an application of said host and a network connecting the host to the plurality of destinations and wherein said network interface includes a queue for each direct memory access connection and adapted to queue messages to be sent through the direct memory access connection associated with each queue, and wherein said each sending of a message to specified memory location of the destination of a direct memory access connection includes queuing the message in the network interface queue associated with the direct memory access connection;

and wherein the queuing of messages in the network interface queue associated with a direct memory access connection is suspended when the quantity of messages sent from said host to the specified memory location of the associated direct memory access connection and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection reaches the separate message limit imposed on the direct memory access connection associated with the network interface queue.

22. The article of manufacture of claim 21, wherein the queuing of messages in the network interface queue associated with a direct memory access connection is resumed when the quantity of messages sent from said host to the specified memory location of the associated direct memory access connection and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection is less than the separate message limit imposed on the direct memory access connection associated with the network interface queue.

23. The article of manufacture of claim 22 wherein each active direct memory access connection includes a Transmission Control Protocol connection between the host and the destination and wherein said first window value is a Transmission Control Protocol send window value and wherein each direct memory access connection is a Remote Direct Memory Access connection between the host and the destination of the direct memory access connection and each message is a Remote Direct Memory Access message.

24. The article of manufacture of claim 23 wherein said network interface has a pool of empty messages which imposes a limit on the total quantity of messages sent from said host to all the specified memory locations of all the direct memory access connections and lacking a message acknowledgment of being received by the destination of the associated direct memory access connection and wherein each message limit is less than the network interface pool of empty messages.

25. The article of manufacture of claim 20 wherein each active direct memory access connection includes a Transmission Control Protocol connection between the host and the destination and wherein said first window value is a Transmission Control Protocol send window value.

26. The article of manufacture of claim 25, wherein the operations further comprise:

in response to the destination reducing the size of the Transmission Control Protocol send window value to a third quantity less than the second quantity, limiting the number of packets sent by said host, but not acknowledged as received by said destination, to a fourth quantity of data packets no greater than the reduced size of the Transmission Control Protocol send window value; and

storing the value of the fourth quantity of data packets in the virtual window field of the associated Protocol Control Block.

27. The article of manufacture of claim 20 further comprising changing the size of a message limit of an active direct memory access connection prior to sending at least one message through the associated direct memory access connection.

28. The article of manufacture of claim 27 wherein each message limit is based, at least in part, on the number of active direct memory access connections of the host.

29. The article of manufacture of claim 20 wherein each message limit is based, at least in part, on the number of active direct memory access connections of the host.

30. The article of manufacture of claim 20 further comprising changing the size of the second quantity of packets limiting the number of packets sent by the host but not acknowledged as received by the destination prior to sending at least one packet.