PARALLEL STREAMS TO MAINTAIN A TARGET DATA TRANSFER RATE

• US 20170004153A1
• Filed: 06/30/2015
• Published: 01/05/2017
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: Spectra Logic Corporation
• Original Assignee: Spectra Logic Corporation
• Inventors: Stevens, Jason Frederick, Carter, Joshua Daniel
• Granted Patent: US 10,063,651 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0682 Tape device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Apparatus and associated method contemplating sending a bulk request to a first archive storage controller to transfer application data from a server at a target transfer rate, the first archive storage controller residing within a complex of archive storage controllers; identifying two or more archive storage controllers in the complex collectively forming a speed matching buffer set capable of parallel streaming the application data at a data transfer rate that is at least the target transfer rate; and as a result of the identifying step, apportioning the application data to the speed matching buffer set.

Claims

1. A method comprising:

sending a bulk request to a first archive storage controller to transfer application data from a server at a target transfer rate, the first archive storage controller residing within a complex of archive storage controllers;

identifying two or more archive storage controllers in the complex collectively forming a speed matching buffer set capable of parallel streaming the application data at a data transfer rate that is at least the target transfer rate; and

as a result of the identifying step, apportioning the application data to the speed matching buffer set.

2. The method of claim 1 further comprising sending parallel data transfers from the server to the speed matching buffer set according to the apportioning step.

3. The method of claim 1 wherein the sending a bulk request step comprises the server sending the bulk request to the first archive storage controller.

4. The method of claim 1 wherein the apportioning step comprises the first archive storage controller sending a bulk response to the server defining the apportionments.

5. The method of claim 1 wherein the identifying step comprises the speed matching buffer set not including the first archive storage controller.

6. The method of claim 1 wherein the identifying step comprises more than two archive storage controllers forming the speed matching buffer set.

7. The method of claim 1 wherein the speed matching buffer set includes fewer than all of the archive storage controllers in the complex, and wherein the identifying step comprises comparing a cache capacity of respective archive storage controllers within the complex.

8. The method of claim 1 wherein the speed matching buffer set includes fewer than all of the archive storage controllers in the complex, and wherein the identifying step comprises comparing the utilization of respective archive storage controllers within the complex.

9. The method of claim 1 wherein the apportioning step comprises apportioning data objects to different archive storage controllers in the speed matching buffer set.

10. The method of claim 1 wherein the apportioning step comprises apportioning different portions of the same data object to different archive storage controllers in the speed matching buffer set.

11. The method of claim 1 wherein each of the archive storage controllers in the speed matching buffer set has a respective cache transfer rate, and wherein the apportioning of the application data is proportional to the cache transfer rate of the respective archive storage controller in the speed matching buffer set.

12. The method of claim 1 wherein the first archive storage controller is a master archive storage controller for the complex, further comprising failing over master control of the complex to another predefined archive storage controller within the complex.

13. A master archive storage controller within a complex of archive storage controllers, the master archive storage controller programmed with computer instructions stored in computer memory that are configured, in response to receiving a bulk request for transferring application data at a target transfer rate, to define a speed matching buffer set of archive storage controllers within the complex that is capable of parallel streaming the application data at the target transfer rate.

14. The storage controller of claim 13 wherein the computer instructions are configured to modify the bulk request by apportioning the application data to the speed matching buffer set.

15. The storage controller of claim 13 wherein the computer instructions are configured to modify the bulk request by not apportioning any of the application data to the first archive storage controller.

16. The storage controller of claim 13 wherein the computer instructions are configured to define the speed matching buffer set by comparing respective cache capacities of the archive storage controllers within the complex.

17. The storage controller of claim 14 wherein the computer instructions are configured to define the speed matching buffer set by comparing respective cache transfer rates of the archive storage controllers within the complex.

18. The storage controller of claim 17 wherein the computer instructions are configured to apportion the application data proportionally to the respective archive storage controller cache transfer rate in the speed matching buffer set.

19. The storage controller of claim 14 wherein the computer instructions are configured to apportion data objects to different archive storage controllers in the speed matching buffer set and to apportion different portions of the same data object to different archive storage controllers in the speed matching buffer set.

20. A method comprising:

sending a bulk request to a first archive storage controller to transfer application data from a server at a target transfer rate, the first archive storage controller residing within a complex of archive storage controllers and having an individual cache transfer rate that is less than the target transfer rate;

identifying two or more archive storage controllers in the complex having individual cache transfer rates that collectively form a speed matching buffer set capable of parallel streaming the application data at a data transfer rate that is at least the target transfer rate;

after the identifying step, apportioning the application data to each archive storage controller in the speed matching buffer set; and

sending parallel data transfers according to the apportioning step to transfer the application data from the server to the speed matching buffer set at the target transfer rate.

21. A method comprising:

sending a bulk request from a client to a first archive storage controller, the first archive storage controller residing within a complex of archive storage controllers;

identifying one or more archive storage controllers in the complex capable of parallel streaming data corresponding to the bulk request based on predefined resource constraints in the complex; and

apportioning the data to the identified archive storage controllers.

22. The method of claim 21 wherein the resource constraints are selected from the group consisting of a faulted archive storage controller, insufficient available primary storage devices, insufficient available secondary storage devices, insufficient available processor capability, and insufficient available communications bandwidth.

23. The method of claim 21 further comprising sending parallel data transfers from the client to the one or more identified archive storage controllers according to the apportioning.

24. The method of claim 23 further comprising issuing a redirect response to the client when the resource constraints change while sending one of the parallel data transfers, the redirect response identifying another archive storage controller for sending an unsent portion of the redirected data transfer.

25. A method comprising:

sending a bulk request to a first archive storage controller;

receiving a plan to send data associated with the bulk request to one or more selected archive storage controllers and acting upon that plan;

upon failure of one of the selected archive storage controllers, identifying one or more non-failed archive storage controllers that are capable of parallel streaming the data based on predefined resource constraints;

as a result of the identifying step, apportioning the data to the non-failed archive storage controllers.