Controlled transfer of data over an elastic network

• US 10,057,327 B2
• Filed: 11/25/2015
• Issued: 08/21/2018
• Est. Priority Date: 11/25/2015
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated (Kyndryl Holdings, Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Chitti, Mark V., Freimuth, Douglas M., Hollingsworth, John F., Mandalia, Baiju D.
• Primary Examiner(s): Dailey, Thomas J
• Application Number:
  US14/952,449
• Publication Number:
  US 20170149863A1
• Time in Patent Office: 1,000 Days
• Field of Search: None
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/622 Queue service order
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements

Abstract

Controlled transfer of data over an elastic network is provided and includes analyzing properties of a set of data for transfer from a source storage location to a target storage location across an elastic cloud computing network as part of a staged transfer of the set of data, assigning transfer queue weightings to data subsets of the set of data based on weighting parameters, the transfer queue weightings dictating order of transfer of the data subsets from the source storage location to the target storage location, and controlling transfer of the data subsets from the source storage location to the target storage location consistent with the transfer queue weightings assigned to the data subsets, the controlling transfer including controlling, based on costs to transfer the data subsets, bandwidth allocation of the elastic cloud computing network and a transfer schedule for the data subsets.

Claims

1. A computer-implemented method comprising:

analyzing properties of a set of data for transfer from a source storage location to a target storage location across an elastic cloud computing network as part of a staged transfer of the set of data, the elastic cloud computing network having controllable elastic network bandwidth for the source storage location and the target storage location;

assigning transfer queue weightings to data subsets of the set of data based on weighting parameters, the transfer queue weightings dictating order of transfer of the data subsets from the source storage location to the target storage location; and

controlling transfer of the data subsets from the source storage location to the target storage location consistent with the transfer queue weightings assigned to the data subsets, the controlling transfer comprising controlling provisioning of wide area network bandwidth of the elastic cloud computing network to at least the source storage location and a transfer schedule for the data subsets, wherein the provisioning of wide area network bandwidth and the transfer schedule are controlled based on costs to transfer the data subsets at different future times, and wherein the provisioning of the wide area network bandwidth is controlled by instructing a dynamic network control application programming interface (API) of a network service provider when and how much bandwidth is to be allocated.

2. The method of claim 1, wherein the data subsets comprise an application and associated application data.

3. The method of claim 2, wherein the properties of the set of data include size of the application and size of the associated application data.

4. The method of claim 1, wherein the weighting parameters comprise one or more of:

data dependencies between the data subsets for transfer, size of the data subsets for transfer, or assigned priority of the data subsets for transfer.

5. The method of claim 1, wherein a weighting parameter of the weighting parameters comprises a rate of change of the set of data or a data subset thereof.

6. The method of claim 1, wherein the costs to transfer the data subsets are dependent at least in part on bandwidth costs, and wherein the provisioned wide area network bandwidth of the elastic cloud computing network and the transfer schedule are controlled based on the bandwidth costs and one or more of:

time to transfer, size of the data subsets, response time requirements, latency, or protocol used to transfer.

7. The method of claim 1, wherein the target storage location comprises an intermediate storage location for the staged transfer, the staged transfer comprising a transfer from the intermediate storage location to a destination storage location.

8. The method of claim 7, wherein the transfer of the data subsets to the target storage location is performed as part of a cloud onboarding procedure, wherein the data subsets are transferred and stored to the intermediate storage location in a different structure than expected for the destination storage location.

9. The method of claim 7, further comprising initiating transfer of the data subsets from the intermediate storage location to the destination storage location for storage at the destination storage location in a structure for runtime usage of one or more applications.

10. A computer program product comprising:

a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

analyzing properties of a set of data for transfer from a source storage location to a target storage location across an elastic cloud computing network as part of a staged transfer of the set of data, the elastic cloud computing network having controllable elastic network bandwidth for the source storage location and the target storage location;

assigning transfer queue weightings to data subsets of the set of data based on weighting parameters, the transfer queue weightings dictating order of transfer of the data subsets from the source storage location to the target storage location; and

controlling transfer of the data subsets from the source storage location to the target storage location consistent with the transfer queue weightings assigned to the data subsets, the controlling transfer comprising controlling provisioning of wide area network bandwidth of the elastic cloud computing network to at least the source storage location and a transfer schedule for the data subsets, wherein the provisioning of wide area network bandwidth and the transfer schedule are controlled based on costs to transfer the data subsets at different future times, and wherein the provisioning of the wide area network bandwidth is controlled by instructing a dynamic network control application programming interface (API) of a network service provider when and how much bandwidth is to be allocated.

11. The computer program product of claim 10, wherein the data subsets comprise an application and associated application data, and wherein the properties of the set of data include size of the application and size of the associated application data.

12. The computer program product of claim 10, wherein a weighting parameter of the weighting parameters comprises a rate of change of the set of data or a data subset thereof.

13. The computer program product of claim 10, wherein the costs to transfer the data subsets are dependent at least in part on bandwidth costs, and wherein the provisioned wide area network bandwidth of the elastic cloud computing network and the transfer schedule are controlled based on the bandwidth costs and one or more of:

time to transfer, size of the data subsets, response time requirements, latency, or protocol used to transfer.

14. The computer program product of claim 10, wherein the target storage location comprises an intermediate storage location for the staged transfer, the staged transfer comprising a transfer from the intermediate storage location to a destination storage location.

15. The computer program product of claim 14, wherein the transfer of the data subsets to the target storage location is performed as part of a cloud onboarding procedure, wherein the data subsets are transferred and stored to the intermediate storage location in a different structure than expected for the destination storage location.

16. The computer program product of claim 14, wherein the method further comprises initiating transfer of the data subsets from the intermediate storage location to the destination storage location for storage at the destination storage location in a structure for runtime usage of one or more applications.

17. A computer system comprising:

a memory; and

a processor in communication with the memory, wherein the computer system is configured to perform a method, the method comprising;

analyzing properties of a set of data for transfer from a source storage location to a target storage location across an elastic cloud computing network as part of a staged transfer of the set of data, the elastic cloud computing network having controllable elastic network bandwidth for the source storage location and the target storage location;

assigning transfer queue weightings to data subsets of the set of data based on weighting parameters, the transfer queue weightings dictating order of transfer of the data subsets from the source storage location to the target storage location; and

controlling transfer of the data subsets from the source storage location to the target storage location consistent with the transfer queue weightings assigned to the data subsets, the controlling transfer comprising controlling provisioning of wide area network bandwidth of the elastic cloud computing network to at least the source storage location and a transfer schedule for the data subsets, wherein the provisioning of wide area network bandwidth and the transfer schedule are controlled based on costs to transfer the data subsets at different future times, and wherein the provisioning of the wide area network bandwidth is controlled by instructing a dynamic network control application programming interface (API) of a network service provider when and how much bandwidth is to be allocated.

18. The computer system of claim 17, wherein a weighting parameter of the weighting parameters comprises a rate of change of the set of data or a data subset thereof.

19. The computer system of claim 17, wherein the costs to transfer the data subsets are dependent at least in part on bandwidth costs, and wherein the provisioned wide area network bandwidth of the elastic cloud computing network and the transfer schedule are controlled based on the bandwidth costs and one or more of:

time to transfer, size of the data subsets, response time requirements, latency, or protocol used to transfer.

20. The computer system of claim 17, wherein the target storage location comprises an intermediate storage location for the staged transfer, the staged transfer comprising a transfer from the intermediate storage location to a destination storage location, wherein the transfer of the data subsets to the target storage location is performed as part of a cloud onboarding procedure, wherein the data subsets are transferred and stored to the intermediate storage location in a different structure than expected for the destination storage location, and wherein the method further comprises initiating transfer of the data subsets from the intermediate storage location to the destination storage location for storage at the destination storage location in a structure for runtime usage of one or more applications.