SYSTEM AND METHOD FOR ADAPTIVE DATA TRANSFERS WITH LIMITED RESOURCES

• US 20160196089A1
• Filed: 01/07/2015
• Published: 07/07/2016
• Est. Priority Date: 01/07/2015
• Status: Abandoned Application

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Gadre, Vijay Vinayak, Reddy CH, Vijaya Bhaskar, Revsin, Vitaly
• Application Number:
  US14/591,557
• Publication Number:
  US 20160196089A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A method and system for adaptive data transfers with limited resources are described. A data storage system performs operations that include migrating data from one set of storage volumes to another set of storage volumes using a number of data streams managed by a storage controller. During this data migration process, the storage controller can measure performance of the data storage system such as the CPU load, memory usage, network bandwidth usage, and the total number of data streams in use. Based on the measured performance and scheduling information, the data storage system can adjust the number of concurrent data streams dedicated to data migration in order to complete the migration process in a timely fashion without substantial undue impact to client accesses and backup operations.

Claims

1. A method of data migration, the method being implemented by one or more processors and comprising:

migrating data from a first set of storage volumes to a second set of storage volumes, wherein migrating data uses a number of concurrent data streams managed by a storage controller;

measuring performance of the storage controller during data migration; and

adjusting the number of concurrent data streams used for migrating data based on the measured performance.

2. The method of claim 1, wherein the first set of storage volumes is associated with a first architecture, and the second set of storage volumes is associated with a second architecture that is different from the first architecture.

3. The method of claim 1, wherein measuring performance of the storage controller includes measuring CPU load, memory usage, network bandwidth usage, number of critical and non-critical data streams in use, and number of available data streams.

4. The method of claim 1, wherein the number of concurrent data streams used for migrating data is increased when the performance of the storage controller is above a first programmed threshold, and wherein the number of concurrent data streams used for migrating data is decreased when the performance of the storage controller is below a second programmed threshold.

5. The method of claim 4, wherein the first and second programmed thresholds are set by an individual with access to the storage controller.

6. The method of claim 1, further comprising:

analyzing a schedule of data streams determined to be more critical than the number of concurrent data streams migrating data; and

reducing the number of concurrent data streams migrating data during times corresponding to the schedule.

7. The method of claim 6, wherein the data streams determined to be more critical than the number of concurrent data streams migrating data are streams that backup data from the first set of storage volumes.

8. A data migration system comprising:

a memory resource to store instructions;

one or more processors using the instructions stored in the memory resource to;

migrate data from a first set of storage volumes to a second set of storage volumes, wherein migrating data uses a number of concurrent data streams managed by a storage controller;

measure performance of the storage controller during data migration; and

adjust the number of concurrent data streams used for migrating data based on the measured performance.

9. The system of claim 8, wherein the first set of storage volumes is associated with a first architecture, and the second set of storage volumes is associated with a second architecture that is different from the first architecture.

10. The system of claim 8, wherein measuring performance of the storage controller includes measuring CPU load, memory usage, network bandwidth usage, number of critical and non-critical data streams in use, and number of available data streams.

11. The system of claim 8, wherein the number of concurrent data streams used for migrating data is increased when the performance of the storage controller is above a first programmed threshold, and wherein the number of concurrent data streams used for migrating data is decreased when the performance of the storage controller is below a second programmed threshold.

12. The system of claim 11, wherein the first and second programmed thresholds are set by an individual with access to the storage controller.

13. The system of claim 8, further comprising instructions to:

analyze a schedule of data streams determined to be more critical than the number of concurrent data streams migrating data; and

reduce the number of concurrent data streams migrating data during times corresponding to the schedule.

14. The system of claim 13, wherein the data streams determined to be more critical than the number of concurrent data streams migrating data are streams that backup data from the first set of storage volumes.

15. A non-transitory computer-readable medium for implementing data migration, the non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations that include:

migrating data from a first set of storage volumes to a second set of storage volumes, wherein migrating data uses a number of concurrent data streams managed by a storage controller;

measuring performance of the storage controller during data migration; and

adjusting the number of concurrent data streams used for migrating data based on the measured performance.

16. The non-transitory computer-readable medium of claim 15, wherein the first set of storage volumes is associated with a first architecture, and the second set of storage volumes is associated with a second architecture that is different from the first architecture.

17. The non-transitory computer-readable medium of claim 15, wherein measuring performance of the storage controller includes measuring CPU load, memory usage, network bandwidth usage, number of critical and non-critical data streams in use, and number of available data streams.

18. The non-transitory computer-readable medium of claim 15, wherein the number of concurrent data streams used for migrating data is increased when the performance of the storage controller is above a first programmed threshold, and wherein the number of concurrent data streams used for migrating data is decreased when the performance of the storage controller is below a second programmed threshold.

19. The non-transitory computer-readable medium of claim 18, wherein the first and second programmed thresholds are set by an individual with access to the storage controller.

20. The non-transitory computer-readable medium of claim 15, further comprising:

analyzing a schedule of data streams determined to be more critical than the number of concurrent data streams migrating data; and

reducing the number of concurrent data streams migrating data during times corresponding to the schedule.