Parallel streams to maintain a target data transfer rate

• US 10,063,651 B2
• Filed: 06/30/2015
• Issued: 08/28/2018
• Est. Priority Date: 06/30/2015
• Status: Active Grant

• Current Assignee: Spectra Logic Corporation
• Original Assignee: Spectra Logic Corporation
• Inventors: Carter, Joshua Daniel, Stevens, Jason Frederick
• Primary Examiner(s): Chea, Philip
• Assistant Examiner(s): Khan, Hassan
• Application Number:
  US14/788,501
• Publication Number:
  US 20170004153A1
• Time in Patent Office: 1,155 Days
• Field of Search: None
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0682 Tape device
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Apparatus and associated method contemplating sending a bulk request to a first archive storage controller to transfer application data from a server at a target transfer rate, the first archive storage controller residing within a complex of archive storage controllers; identifying two or more archive storage controllers in the complex collectively forming a speed matching buffer set capable of parallel streaming the application data at a data transfer rate that is at least the target transfer rate; and as a result of the identifying step, apportioning the application data to the speed matching buffer set.

Claims

1. A method for archiving data from a server cluster to a backend data storage device via a computer network, the method comprising:

placing a complex of archive storage controllers in the network, the complex including a master archive storage controller having a speed matching buffer set module;

the server cluster executing backup logic programmed to send a bulk input/output (I/O) request for a greater than a predetermined-size transfer of archive data to the master archive storage controller; and

the speed matching buffer set module programmed, responsive to receiving the bulk I/O request, to identify a target transfer rate as defined by the bulk I/O request or by a predetermined relationship to I/O load, to identify identifying a speed matching buffer set of two or more of the archive storage controllers in the complex that are available, for lack of a predefined resource constraint characterized in terms of operational state, data storage capacity, processor capability, bandwidth, or utilization, to collectively transfer the bulk I/O request at the target transfer rate, and to send a bulk I/O response used by the server cluster to modify the bulk I/O request by apportioning it into parallel I/O requests to the speed matching buffer set.

2. The method of claim 1 further comprising the backup logic, responsive to receiving the bulk I/O response, sending the parallel I/O requests to the speed matching buffer set.

3. The method of claim 1 wherein the identifying the speed matching buffer set step does not include the master archive storage controller in the speed matching buffer set.

4. The method of claim 1 wherein the identifying the speed matching buffer set step comprises including more than two archive storage controllers in the speed matching buffer set.

5. The method of claim 1 wherein the speed matching buffer set includes fewer than all of the archive storage controllers in the complex, and wherein the identifying the speed matching buffer set step comprises comparing a cache capacity of respective archive storage controllers within the complex.

6. The method of claim 1 wherein the speed matching buffer set includes fewer than all of the archive storage controllers in the complex, and wherein the identifying the speed matching buffer set step comprises comparing the utilization of respective archive storage controllers within the complex.

7. The method of claim 1 wherein the modifying step comprises apportioning data objects to different archive storage controllers in the speed matching buffer set.

8. The method of claim 1 wherein the modifying step comprises apportioning different portions of the same data object to different archive storage controllers in the speed matching buffer set.

9. The method of claim 1 wherein each of the archive storage controllers in the speed matching buffer set has a respective cache transfer rate, and wherein the modifying step apportions the application data proportionally to the cache transfer rate of the respective archive storage controller in the speed matching buffer set.

10. The method of claim 1 wherein the further comprising failing over master control of the complex to another predefined archive storage controller within the complex.

11. A non-transitory distributed data storage system, comprising:

a server cluster operably executing backup logic for computer applications via input/output (I/O) commands over a computer network;

a backend data storage device configured to store archive data generated by the backup logic;

a complex of archive storage controllers processing the I/O commands to transfer the archive data from the server cluster to the backend data storage device via the computer network, the complex including a master archive storage controller to which the backup logic sends a bulk I/O request for a greater than a predetermined-size data transfer, the master archive storage controller having a speed matching buffer set module programmed, in response to the bulk I/O request, to identify a target transfer rate as defined by the bulk I/O request or by a predefined relationship to I/O load, to identify a speed matching buffer set of two or more of the archive storage controllers in the complex that are available, for lack of a predefined resource constraint in terms of operational state, data storage capacity, processor capability, bandwidth, or utilization, to collectively transfer the bulk I/O request at the target transfer rate, and to send a bulk I/O response that is used by the server cluster to modify the bulk I/O request by apportioning it into parallel I/O requests to the speed matching buffer set.

12. The system of claim 11 wherein the speed matching buffer set module is programmed to modify the bulk request by not apportioning any of the application data to the master archive storage controller.

13. The system of claim 11 wherein the speed matching buffer set module is programmed to define the speed matching buffer set by comparing respective cache capacities of the archive storage controllers within the complex.

14. The system of claim 11 wherein the speed matching buffer set module is programmed to define the speed matching buffer set by comparing respective cache transfer rates of the archive storage controllers within the complex.

15. The system of claim 14 wherein the speed matching buffer set module is programmed to apportion the application data proportionally to the respective archive storage controller cache transfer rate in the speed matching buffer set.

16. The system of claim 11 wherein the speed matching buffer set module is programmed to apportion data objects to different archive storage controllers in the speed matching buffer set and to apportion different portions of the same data object to different archive storage controllers in the speed matching buffer set.

17. The system of claim 11 wherein the speed matching buffer set module is programmed to select the speed matching buffer set based on resource constraints in the complex selected from the group consisting of a faulted archive storage controller, insufficient available primary storage devices, insufficient available secondary storage devices, insufficient available processor capability, and insufficient available communications bandwidth.

18. A non-transitory complex of archive storage controllers for processing input/output (I/O) commands from a server cluster operably executing backup logic for computer applications to store archive data generated by the backup logic with a backend data storage device via a computer network, the complex comprising a master archive storage controller to which the backup logic sends a bulk I/O request for a greater than a predetermined-size data transfer, the master archive storage controller having a speed matching buffer set module programmed, in response to receiving the bulk I/O request, to identify a target transfer rate as defined by the bulk I/O request or by a predefined relationship to I/O load, to identify a speed matching buffer set of two or more of the archive storage controllers in the complex that are available, for lack of a predefined resource constraint in terms of operational state, data storage capacity, processor capability, bandwidth, or utilization, to collectively transfer the bulk I/O request at the target transfer rate, and to send a bulk I/O response used by the server cluster to modify the bulk I/O request by apportioning it into parallel I/O requests to the speed matching buffer set.