DATA STORAGE RESOURCE ALLOCATION BY EMPLOYING DYNAMIC METHODS AND BLACKLISTING RESOURCE REQUEST POOLS

• US 20090320033A1
• Filed: 06/19/2008
• Published: 12/24/2009
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Karandikar, Amey, Gokhale, Parag, Wang, Yu, Kottomtharayil, Rajiv
• Granted Patent: US 9,128,883 B2
• Time in Patent Office: Days
• US Class Current: 718/103
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4818 Priority circuits therefor
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 47/805 QOS or priority aware
• CPC Class Codes: H04L 63/101 Access control lists [ACL]

Abstract

A resource allocation system begins with an ordered plan for matching requests to resources that is sorted by priority. The resource allocation system optimizes the plan by determining those requests in the plan that will fail if performed. The resource allocation system removes or defers the determined requests. In addition, when a request that is performed fails, the resource allocation system may remove requests that require similar resources from the plan. Moreover, when resources are released by a request, the resource allocation system may place the resources in a temporary holding area until the resource allocation returns to the top of the ordered plan so that lower priority requests that are lower in the plan do not take resources that are needed by waiting higher priority requests higher in the plan.

Claims

1. A method of matching data storage requests to data storage resources, the method comprising:

receiving a list of data storage requests to be performed by a data storage system, wherein each data storage request specifies two or more data storage resources that are required to perform the data storage request;

without having to perform a first data storage request itself, performing one or more abbreviated checks to determine whether the first data storage request will fail, wherein performing one or more abbreviated checks includes performing a selected subset of one or more checks for whether at least a selected one of the data storage resources are available to satisfy the first data storage request, and wherein the selected one data storage resource is a physical data storage resource, rather than a logical data storage resource; and

when the one or more abbreviated checks indicate that the first data storage request would fail if performed, then;

updating the received list of data storage requests to indicate that the first data storage request should not be performed, without attempting to perform the first data storage request and without performing a check of all data storage resources required for performing the data storage request; and

again performing the one or more abbreviated checks on a second data storage request in the received list of data storage requests, and updating the received list when the one or more abbreviated checks indicate that the second data storage request would fail if performed.

2. The method of claim 1 wherein performing one or more abbreviated checks comprises determining whether two or more of the following data storage resources are available:

data storage drives, device streams, data storage media, data streams, data storage resources based on an allocation policy, and access to a data storage library.

3. The method of claim 1 wherein performing one or more abbreviated checks comprises determining a load on data storage hardware.

4. The method of claim 1 wherein performing one or more abbreviated checks further comprises performing logical checks, including determining whether a drive is busy or flagged not to be used.

5. The method of claim 1 wherein performing one or more abbreviated checks comprises determining whether one or more data storage devices are reserved for other uses.

6. The method of claim 1 wherein the list of data storage requests is ordered by a priority assigned to performing each data storage request.

7. A system for handling a prioritized queue of data management requests, the system comprising:

a receive requests component configured to receive a queue of data management requests, wherein each data management request is to perform at least one data management operation using data management resources from a pool of data management resources, and wherein the queue includes a prioritized list of data management requests;

a handle request component configured to communicate with the receive requests component and to at least initiate each data management request, wherein the handle request component attempts to handle each data management request in order based on the prioritized list of data management requests if data management resources from the pool of data management resources are available; and

a resource store component configured to reserve one or more data management resources released by an at least partially completed data management request, and

wherein the handle request component is configured to not provide the released one or more data management resources to a first received but lower priority data management request, but instead to provide the released one or more data management resources to a later received data management request having a higher priority than the lower priority data management request.

8. The system of claim 7, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria, and wherein at least one optimization criteria is an abbreviated check to determine if a data management request would fail, and wherein if the data management request would fail or does fail, removing the data management request from the queue of data management requests.

9. The system of claim 7, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria, and wherein at least one optimization criteria includes a blacklisted criteria that identifies at least one category of data management requests based on the data management resources used by the data management requests, and wherein if a data management request does fail or would fail, removing other data management requests having the same category from the queue of data management requests.

10. The system of claim 7, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria, and wherein at least one optimization criteria is a table of alternative resources that match data management requests to data management resources based on a user specified categorization of the data management resources.

11. The system of claim 7 wherein the resource store component returns the one or more resources to the global pool of resources after one or more lower priority data management requests are at least partially completed.

12. The system of claim 7 wherein the resource store component returns the one or more resources to the global pool of resources rather than reserving the released one or more resources if a threshold number or percentage of the data management requests in the prioritized list of data management requests have been completed.

13. The system of claim 7, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria.

14. A computer-readable medium encoded with instructions for controlling a computer system to perform a set of data management requests within a selected time period, by a method comprising:

receiving a list of data management requests to be performed by a data management system, wherein each data management request is to perform at least one data management operation using data management resources;

attempting to perform a data management request from the list of data management requests by assigning the data management request to one or more available data management resources;

if the data management request fails,

determining at least one data management resource at least partially responsible for the failure;

determining a category associated with the one data management resource at least partially responsible for the failure, wherein data management resources are associated with categories;

identifying other data management requests in the list of data management requests that request data management resources having the same category; and

updating the received list of data management requests to indicate that the data management system should not perform the other identified data management requests.

15. The computer-readable medium of claim 14 wherein a category is created for each common resource that is likely to be critical to the performance of a data management request.

16. The computer-readable medium of claim 14 wherein identifying other data management requests in the list of data management requests comprises determining a likeness factor among the data management requests that use similar data management resources, and treating the categories as the same when the likeness factor exceeds a threshold.

17. The computer-readable medium of claim 14 wherein identifying other data management requests in the list of data management requests comprises determining whether the data management request failed because of a common resource, and when the data management request fails because of a common resource, then identifying other data management requests in the list of data management requests that request data management resources having the same category as the common resource.

18. The computer-readable medium of claim 14 wherein identifying other data management requests in the list of data management requests comprises determining whether the data management request failed because of a unique resource, and when the data management request fails because of a unique resource, then not identifying other data management requests in the list of data management requests that request data management resources having the same category as on the unique resource.