METHOD AND SYSTEM FOR I/O DRIVEN RATE ADAPTATION

• US 20100228878A1
• Filed: 03/05/2010
• Published: 09/09/2010
• Est. Priority Date: 03/06/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Aspera, Inc. (International Business Machines Corporation)
• Inventors: Xu, Ying, Munson, Michelle Christine, Simu, Serban
• Granted Patent: US 8,719,443 B2
• Time in Patent Office: Days
• US Class Current: 709/233
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 47/18 End to end
• CPC Class Codes: H04L 47/25 with rate being modified by the source upon detecting a change of network conditions
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 47/30 in combination with information about buffer occupancy at either end or at transit nodes
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: Y02D 30/50 in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

Described is an I/O driven, feedback based rate control mechanism that allows a transfer session to self-adjust its transfer rate in response to the speed of the storage device at the destination. The described system may operate at the application layer, which makes cross-platform deployment an easier task

Claims

1. A system, comprising:

a first host and a second host connected over a network, the second host being connected to a storage device;

a sending process executable in the first host for communicating data to a receiving process executable in the second host in order for the receiving process to write the data to the storage device, wherein the sending process is configured to send data to the receiving process at a specified variable data transfer rate x(t) at any time t, wherein the receiving process is configured to store data received in a ring buffer before writing it to the storage device;

a storage rate controller incorporated into the receiving process, wherein the storage rate controller is configured to;

     1) measure the current occupancy of the ring buffer,

     2) calculate an updated data transfer rate x′

(t) as a function of the current data transfer rate x(t), the measured current occupancy, the total ring buffer size, and a specified target occupancy value, and

     3) communicate the updated data transfer rate to the sending process; and

,

wherein the sending process is configured to adjust the data transfer rate x(t) to the updated data transfer rate x′

(t).

2. The system of claim 1 wherein the storage rate controller is configured such that the updated data transfer rate x′

(t) represents a decrease of the current data transfer rate x(t) if the measured current occupancy is greater than the specified target occupancy value.

3. The system of claim 1 wherein the rate controller is configured to compute the updated data transfer rate x′

(t) after finishing a write operation to the storage device.

4. The system of claim 1 wherein the rate controller is configured to compute the updated data transfer rate x′

(t) at periodic intervals.

5. The system of claim 1 wherein the rate controller is configured to:

1) after finishing a write operation to the storage device, compute the updated data transfer rate x′

(t) from a first function of the current data transfer rate x(t), the measured current buffer occupancy, the total size of the ring buffer, and a specified target occupancy value, and

     2) at periodic intervals, compute the updated data transfer rate x′

(t) from a second function of the current data transfer rate x(t), the measured current buffer occupancy, the total size of the ring buffer, and a specified target occupancy value.

6. The system of claim 4 wherein the periodic intervals correspond to a currently measured network round-trip time RTT(t).

7. The system of claim 4 wherein the periodic intervals correspond to a currently measured network round-trip time RTT(t) if RTT(t) is below a specified threshold value and correspond to a specified default value otherwise.

8. The system of claim 5 wherein the storage rate controller is configured to compute the updated data transfer rate x′

(t) from a first function as;

9. The system of claim 5 wherein the storage rate controller is configured to compute the updated data transfer rate x′

(t) from a first function as;

10. The system of claim 5 wherein the storage rate controller is configured to compute the updated data transfer rate x′

(t) at periodic intervals equal to RTT(t) from a second function as;

x′

(t)=α

(t)x(t) ρ

(t)≧

ρ

_min

α

(t)=[1−

ρ

(t)]/[1−

ρ

′

]

when the current occupancy ρ

(t) is greater than or equal to a predetermined value ρ

_min, where ρ

′

is a specified target occupancy.

11. The system of claim 1 further comprising a plurality of sending and receiving processes, with each such receiving process being connected to the storage device and incorporating a storage rate controller and a ring buffer for updating the data transfer rate of its respective sending process.

12. The system of claim 1 wherein the receiving process further incorporates a network rate controller configured to measure an extent of network congestion and to compute a network rate x_net(t) based on the measured congestion, and wherein the sending process further adjust the current data transfer rate as a function of x_net(t) and the updated data transfer rate sent by the receiving process.

13. The system of claim 12 wherein the measure of network congestion is a queuing delay derived from a measured round trip delay time.

14. The system of claim 1 wherein the sending and receiving processes belong to a group of similar processes that together limit their aggregate data transfer rate by maintaining a virtual queue and calculating an occupancy of the virtual queue based upon their aggregate data transfer rate, and wherein each receiving process further incorporates a virtual link controller which computes a virtual link rate x_vl(t) based on the virtual queue occupancy, and wherein each sending process adjusts its data transfer rate as a function of the virtual link rate x_vl(t) and the updated data transfer rate x′

(t) sent by the receiving process.

15. The system of claim 1 wherein the receiving process further incorporates a network rate controller configured to measure an extent of network congestion, and wherein the sending process further adjusts its data transfer rate as a function of a network rate x_net(t) calculated from the measured network congestion, the virtual link control rate x_vl(t) calculated from the virtual queue occupancy, and the updated data transfer rate x′

(t) sent by the receiving process using a minimal selection criterion as;

x(t)=min{x_net(t),x_vl(t),x′

(t)}.

16. The system of claim 1 wherein the storage rate controller is configured to compute the updated data transfer rate x′

(t) as;

17. The system of claim 1 wherein the storage rate controller is configured to compute the updated data transfer rate x′

(t) as;

18. A method for transferring data from a first host to a second host connected over a network, the second host being connected to a storage device, comprising:

executing a sending process in the first host for communicating data to a receiving process executing in the second host in order for the receiving process to write the data to the storage device, wherein the sending process sends data to the receiving process at a specified variable data transfer rate x(t) at any time t, and wherein the receiving process stores data received in a ring buffer before writing it to the storage device;

executing a storage rate controller incorporated into the receiving process, to;

     1) measure the current occupancy of the ring buffer,

     2) calculate an updated data transfer rate x′

(t) as a function of the current data transfer rate x(t), the measured current occupancy, and a specified target occupancy value, and

     3) communicate the updated data transfer rate to the sending process; and

,

adjusting the data transfer rate x(t) to the updated data transfer rate x′

(t).

19. The method of claim 18 further comprising:

after finishing a write operation to the storage device, computing the updated data transfer rate x′

(t) from a first function of the current data transfer rate x(t), the measured current buffer occupancy, the total size of the ring buffer, and a specified target occupancy value; and

at periodic intervals, computing the updated data transfer rate x′

(t) from a second function of the current data transfer rate x(t), the measured current buffer occupancy, the total size of the ring buffer, and a specified target occupancy value.

20. The method of claim 19 further comprising computing the updated data transfer rate x′

(t) from a first function as;