Adaptive performance control of multi-stream data transfer from a source system to a target system

• US 9,509,612 B1
• Filed: 06/30/2015
• Issued: 11/29/2016
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Armorer, Peter, Muchalambe, Uday Devidas, Fernandes, Roland Anthony, Kulkarni, Gururaj, Chai, Mu
• Primary Examiner(s): Orgad, Edan
• Assistant Examiner(s): SONG, REBECCA E
• Application Number:
  US14/755,292
• Time in Patent Office: 518 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04L 43/0888 Throughput
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/41 by acting on aggregated flows or links
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/75 Media network packet handling

Abstract

According to one embodiment, in response to a request to transfer a data set from a source system to a target system over a network, an adaptive performance control (APC) controller allocates a plurality of data streams for transferring the data set. The APC controller activates one or more data streams from the allocated data streams to transfer the data set from the source system to the target system. The APC controller monitors an overall throughput of the activated data streams assigned to transfer the data set from the source system to the target system. The APC controller dynamically adjusts a number of the activated data streams based on the monitored overall throughput of the activated data streams, such that a maximum overall throughput of the activated data streams is reached while maintaining a minimum number of the activated data streams.

Claims

1. A computer-implemented method for managing data streams for data transfer operations, the method comprising:

in response to a request to transfer a data set from a source system to a target system over a network, allocating, by an adaptive performance control (APC) controller executed by a processor, a predetermined number of data streams that, when activated, actively transfer at least a portion of the data set from the source system to the target system;

activating by the APC controller a primary data stream and at least one secondary data stream from the allocated data streams to transfer the data set from the source system to the target system, wherein the primary data stream remains activated during the transfer of the data set, and a secondary data stream is activated, deactivated, or reactivated during the transfer of the data set;

monitoring by the APC controller an overall throughput of the activated data streams assigned to transfer the data set from the source system to the target system;

activating another secondary data stream of the allocated data streams;

monitoring the overall throughput of the activated data streams for a first predetermined period of time; and

dynamically adjusting by the APC controller a number of the activated data streams based on the monitored overall throughput of the activated data streams, such that a maximum overall throughput of the activated data streams is reached while maintaining a minimum number of the activated data streams, wherein dynamically adjusting, by the APC, the number of activated data streams further comprises;

monitoring the overall throughput of the activated data streams for a second predetermined period of time;

deactivating a secondary data stream from the activated data streams if the overall throughput drops below a second predetermined threshold over the second predetermined period of time;

determining whether all allocated data streams have been activated if the overall throughput does not drop below the second predetermined threshold;

iteratively performing monitoring the overall throughput for the second predetermined period of time and deactivating a secondary data stream, in response to determining that all allocated data streams have been activated;

suspending operation of the APC for a third predetermined period of time in response to determining that the throughput drop is not greater than the second predetermined threshold and there is at least one allocated data stream that has been previously deactivated; and

reactivating a fourth data stream that has been previously deactivated after the third predetermined period of time.

2. The method of claim 1, wherein dynamically adjusting, by the APC, the number of activated data streams comprises:

deactivating a secondary data stream from the activated data streams, in response to determining that the overall throughput has gained less than the first predetermined threshold by the activating of the another secondary data stream.

3. The method of claim 1, wherein dynamically adjusting, by the APC, the number of activated streams further comprises:

iteratively performing activating another secondary data stream and monitoring the overall throughput for the first predetermined period of time, if the overall throughput has gained more than or equal to the first predetermined threshold.

4. The method of claim 3, wherein dynamically adjusting, by the APC, the number of activated streams further comprises:

determining whether all allocated data streams have been activated, wherein iteratively performing activating another secondary data stream and monitoring the overall throughput for the first predetermined period of time is performed only if there is at least one allocated data stream that has not been activated.

5. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of managing data streams for data transfer operations, the operations comprising:

in response to a request to transfer a data set from a source system to a target system over a network, allocating a predetermined number of data streams the data set;

activating a primary data stream and at least one secondary data stream from the allocated data streams to transfer the data set from the source system to the target system, wherein the primary data stream remains activated during the transfer of the data set, and a secondary stream is activated, deactivated, or reactivated, during the transfer of the data set;

monitoring an overall throughput of the activated data streams assigned to transfer the data set from the source system to the target system;

activating another secondary data stream of the allocated data streams;

monitoring the overall throughput of the activated data streams for a first predetermined period of time; and

dynamically adjusting a number of the activated data streams based on the monitored overall throughput of the activated data streams, such that a maximum overall throughput of the activated data streams is reached while maintaining a minimum number of the activated data streams, wherein dynamically adjusting, by the APC, the number of activated data streams further comprises;

monitoring the overall throughput of the activated data streams for a second predetermined period of time;

deactivating a secondary data stream from the activated data streams if the overall throughput drops below a second predetermined threshold over the second predetermined period of time;

determining whether all allocated data streams have been activated if the overall throughput does not drop below the second predetermined threshold;

iteratively performing monitoring the overall throughput for the second predetermined period of time and deactivating a secondary data stream, in response to determining that all allocated data streams have been activated;

suspending operation of the APC for a third predetermined period of time in response to determining that the throughput drop is not greater than the second predetermined threshold and there is at least one allocated data stream that has been previously deactivated; and

reactivating a fourth data stream that has been previously deactivated after the third predetermined period of time.

6. The non-transitory machine-readable medium of claim 5, wherein dynamically adjusting the number of activated data streams comprises:

deactivating a secondary data stream from the activated data streams, in response to determining that the overall throughput has gained less than the first predetermined threshold by the activating of the another secondary data stream.

7. The non-transitory machine-readable medium of claim 5, wherein dynamically adjusting the number of activated streams further comprises:

iteratively performing activating another secondary data stream and monitoring the overall throughput for the first predetermined period of time, if the overall throughput has gained more than or equal to the first predetermined threshold.

8. The non-transitory machine-readable medium of claim 7, wherein dynamically adjusting the number of activated data streams further comprises:

determining whether all allocated data streams have been activated, wherein iteratively performing activating another secondary data stream and monitoring the overall throughput for the first predetermined period of time is performed only if there is at least one allocated data stream that has not been activated.

9. A data processing system operating as a source system, comprising:

a processor; and

a memory storing instructions, which when executed by the processor, cause the processor to perform operations, the operations including

in response to a request to transfer a data set from the source system to a target system over a network, allocating a predetermined number of data streams that, when activated, actively transfer at least a portion of the data set,

activating a primary data stream and at least one secondary data stream from the allocated data streams to transfer the data set from the source system to the target system, wherein the primary data stream remains activated during the transfer of the data set, and a secondary data stream is activated, deactivated, or reactivated during the transfer of the data set,

monitoring an overall throughput of the activated data streams assigned to transfer the data set from the source system to the target system,

activating another secondary data stream of the allocated data streams,

monitoring the overall throughput of the activated data streams for a first predetermined period of time, and

dynamically adjusting a number of the activated data streams based on the monitored overall throughput of the activated data streams, such that a maximum overall throughput of the activated data streams is reached while maintaining a minimum number of the activated data streams, wherein dynamically adjusting, by the APC, the number of activated data streams further comprises;

monitoring the overall throughput of the activated data streams for a second predetermined period of time;

deactivating a secondary data stream from the activated data streams if the overall throughput drops below a second predetermined threshold over the second predetermined period of time;

determining whether all allocated data streams have been activated if the overall throughput does not drop below the second predetermined threshold;

iteratively performing monitoring the overall throughput for the second predetermined period of time and deactivating a secondary data stream, in response to determining that all allocated data streams have been activated;

suspending operation of the APC for a third predetermined period of time in response to determining that the throughput drop is not greater than the second predetermined threshold and there is at least one allocated data stream that has been previously deactivated; and

reactivating a fourth data stream that has been previously deactivated after the third predetermined period of time.

10. The system of claim 9, wherein dynamically adjusting the number of activated data streams further comprises:

deactivating a secondary data stream from the activated data streams, in response to determining that the overall throughput has gained less than the first predetermined threshold by the activating of the another data stream.

11. The system of claim 9, wherein dynamically adjusting the number of activated data streams further comprises:

iteratively performing activating another secondary data stream and monitoring the overall throughput for the first predetermined period of time, if the overall throughput has gained more than or equal to the first predetermined threshold.

12. The system of claim 11, wherein dynamically adjusting the number of activated data streams further comprises:

determining whether all allocated data streams have been activated, wherein iteratively performing activating another secondary data stream and monitoring the overall throughput for the first predetermined period of time is performed only if there is at least one allocated data stream that has not been activated.