Data storage resource allocation by performing abbreviated resource checks based on relative chances of failure of the data storage resources to determine whether data storage requests would fail

• US 9,128,883 B2
• Filed: 06/19/2008
• Issued: 09/08/2015
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv, Karandikar, Amey, Wang, Yu
• Primary Examiner(s): Huisman, David J
• Application Number:
  US12/142,423
• Publication Number:
  US 20090320033A1
• Time in Patent Office: 2,637 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4818 Priority circuits therefor
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 47/805 QOS or priority aware
• CPC Class Codes: H04L 63/101 Access control lists [ACL]

Abstract

A resource allocation system begins with an ordered plan for matching requests to resources that is sorted by priority. The resource allocation system optimizes the plan by determining those requests in the plan that will fail if performed. The resource allocation system removes or defers the determined requests. In addition, when a request that is performed fails, the resource allocation system may remove requests that require similar resources from the plan. Moreover, when resources are released by a request, the resource allocation system may place the resources in a temporary holding area until the resource allocation returns to the top of the ordered plan so that lower priority requests that are lower in the plan do not take resources that are needed by waiting higher priority requests higher in the plan.

Claims

1. A method of performing abbreviated resource checks to determine whether data storage requests would fail in a data storage system, the method comprising:

receiving a list of data storage requests to be performed by the data storage system,

wherein the data storage requests in the list form part of a data backup operation, and

wherein each data storage request in the list specifies two or more data storage resources that are required to perform the respective data storage request;

for a first data storage request, without having to perform the first data storage request itself, performing one or more abbreviated checks to determine whether the first data storage request will fail,

wherein performing the one or more abbreviated checks includes;

(i) determining which of the two or more data storage resources specified by the first data storage request are physical resources,

(ii) selecting the data storage resources determined to be physical resources, and

(iii) performing, in an order based on relative chances of failure of the respective selected physical resources, one or more physical checks of whether the selected physical resources are available to satisfy the first data storage request, the performing the one or more physical checks including determining whether hardware for performing the first data storage request is working,

wherein the one or more physical checks of the selected physical resources are performed before any logical checks of the selected physical resources may be performed; and

when the one or more abbreviated checks indicate that the first data storage request would fail if performed, then;

updating the received list of data storage requests by removing the first data storage request from the received list of data storage requests to indicate that the first data storage request should not be performed,

wherein the updating is performed without attempting to perform the first data storage request and without performing a check of all data storage resources required for performing the first data storage request.

2. The method of claim 1 wherein performing the one or more abbreviated checks comprises determining whether two or more of the following data storage resources are available:

data storage drives, device streams, data storage media, data streams, data storage resources based on an allocation policy, and access to a data storage library.

3. The method of claim 1 wherein performing the one or more abbreviated checks comprises determining a load on data storage hardware.

4. The method of claim 1 wherein the updating of the received list of data storage requests by removing the first data storage request further comprises removing from the received list at least one other data storage request that depends from the first data storage request.

5. The method of claim 1 wherein performing the one or more abbreviated checks comprises determining whether one or more data storage devices are reserved for other uses.

6. The method of claim 1 wherein the list of data storage requests is ordered by a priority assigned to performing each data storage request.

7. The method of claim 1 wherein checking those of the selected physical resources that are deemed to have a higher chance of failure occurs before checking others of the selected physical resources which are deemed to have a relatively lower chance of failure.

8. The method of claim 1 wherein the relative chance of failure of a given physical resource is determined based on the scarcity of that type of physical resource in the data storage system.

9. A non-transitory computer-readable storage medium storing instructions, which when executed by at least one data processor in a data storage system, perform a method of performing abbreviated resource checks to determine whether data storage requests would fail, the method comprising:

receiving a list of data storage requests to be performed by the data storage system,

wherein the data storage requests in the list form part of a data backup operation, and

wherein each data storage request in the list specifies two or more data storage resources that are required to perform the respective data storage request;

for a first data storage request, without having to perform the first data storage request itself, performing one or more abbreviated checks to determine whether the first data storage request will fail,

wherein performing the one or more abbreviated checks includes;

(i) determining which of the two or more data storage resources specified by the first data storage request are physical resources,

(ii) selecting the data storage resources determined to be physical resources, and

(iii) performing, in an order based on relative chances of failure of the respective selected physical resources, one or more physical checks of whether the selected physical resources are available to satisfy the first data storage request, the performing the one or more physical checks including determining whether hardware for performing the first data storage request is working,

wherein the one or more physical checks of the selected physical resources are performed before any logical checks of the selected physical resources may be performed; and

when the one or more abbreviated checks indicate that the first data storage request would fail if performed, then;

updating the received list of data storage requests by removing the first data storage request from the received list of data storage requests to indicate that the first data storage request should not be performed,

wherein the updating is performed without attempting to perform the first data storage request and without performing a check of all data storage resources required for performing the first data storage request.

10. The non-transitory computer-readable storage medium of claim 9 wherein performing the one or more abbreviated checks comprises determining whether two or more of the following data storage resources are available:

data storage drives, device streams, data storage media, data streams, data storage resources based on an allocation policy, and access to a data storage library.

11. The non-transitory computer-readable storage medium of claim 9 wherein performing the one or more abbreviated checks comprises determining a load on data storage hardware.

12. The non-transitory computer-readable storage medium of claim 9 wherein the updating of the received list of data storage requests by removing the first data storage request further comprises removing from the received list at least one other data storage request that depends from the first data storage request.

13. The non-transitory computer-readable storage medium of claim 9 wherein performing the one or more abbreviated checks comprises determining whether one or more data storage devices are reserved for other uses.

14. The non-transitory computer-readable storage medium of claim 9 wherein the list of data storage requests is ordered by a priority assigned to performing each data storage request.

15. The non-transitory computer-readable storage medium of claim 9 wherein those of the selected physical resources that are deemed to have a higher chance of failure are checked before others of the selected physical resources which are deemed to have a relatively lower chance of failure.

16. The non-transitory computer-readable storage medium of claim 9 wherein the relative chance of failure of a given physical resource is determined based on the scarcity of that type of physical resource in the data storage system.

17. A system for matching data storage requests to data storage resources, the system comprising:

a server for performing abbreviated resource checks to determine whether data storage requests would fail, wherein the server is configured to;

receive a list of data storage requests to be performed by the system,

wherein the data storage requests in the list form part of a data backup operation, and

wherein each data storage request in the list specifies two or more data storage resources in the system that are required to perform the data storage request;

for a first data storage request, without having to perform the first data storage request itself, perform one or more abbreviated checks to determine whether the first data storage request will fail,

wherein to perform the one or more abbreviated checks the server is further configured to;

(i) determine which of the two or more data storage resources specified by the first data storage request are physical resources,

(ii) select the data storage resources determined to be physical resources, and

(iii) in an order based on relative chances of failure of the respective selected physical resources, perform one or more physical checks of whether the selected physical resources are available to satisfy the first data storage request, which includes determining whether hardware for performing the first data storage request is working, and

 

wherein the server is configured to perform the one or more physical checks before it may perform any logical checks;

when a physical check of a selected physical resource indicates that the first data storage request would fail if performed, remove the first data storage request from the received list of data storage requests to indicate that the first data storage request should not be performed, without attempting to perform the first data storage request and without performing a check of all data storage resources required for performing the first data storage request; and

after performing the one or more abbreviated checks on the first data storage request, (a) again perform the one or more abbreviated checks on a second data storage request in the received list of data storage requests, and (b) remove the second data storage request from the received list when the one or more abbreviated checks indicate that the second data storage request would fail if performed.

18. The system of claim 17 wherein to perform the one or more abbreviated checks, the server is further configured to determine whether two or more of the following data storage resources are available:

data storage drives, device streams, data storage media, data streams, data storage resources based on an allocation policy, and access to a data storage library.

19. The system of claim 17 wherein to perform the one or more abbreviated checks, the server is further configured to determine a load on data storage hardware.

20. The system of claim 17 wherein to perform the one or more abbreviated checks, the server is further configured to determine whether one or more data storage devices are reserved for other uses.