DATA STORAGE RESOURCE ALLOCATION USING BLACKLISTING OF RESOURCE REQUEST POOLS SUCH AS CATEGORIES OF DATA STORAGE REQUESTS

• US 20150324233A1
• Filed: 07/21/2015
• Published: 11/12/2015
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv, Karandikar, Amey Vijaykumar, Wang, Yu
• Granted Patent: US 9,612,916 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4818 Priority circuits therefor
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 47/805 QOS or priority aware
• CPC Class Codes: H04L 63/101 Access control lists [ACL]

Abstract

A resource allocation system begins with an ordered plan for matching requests to resources that is sorted by priority. The resource allocation system optimizes the plan by determining those requests in the plan that will fail if performed. The resource allocation system removes or defers the determined requests. In addition, when a request that is performed fails, the resource allocation system may remove requests that require similar resources from the plan. Moreover, when resources are released by a request, the resource allocation system may place the resources in a temporary holding area until the resource allocation returns to the top of the ordered plan so that lower priority requests that are lower in the plan do not take resources that are needed by waiting higher priority requests higher in the plan.

Claims

1. A computer-readable medium encoded with instructions for controlling a computer system to perform a set of data management requests within a selected time period, by a method comprising:

receiving a list of data management requests to be performed by a data management system,

wherein each data management request is to perform at least one data management operation using data management resources;

attempting to perform a data management request from the list of data management requests by assigning the data management request to one or more available data management resources;

if the data management request fails, then;

identifying at least one data management resource at least partially responsible for the failure;

determining a data management resource category associated with the one data management resource at least partially responsible for the failure,

wherein each data management resource is associated with a data management resource category;

identifying other data management requests in the list of data management requests that request data management resources having the same category; and

updating the received list of data management requests to indicate that the data management system is not to perform the other identified data management requests.

2. The computer-readable medium of claim 1 wherein a category is created for each common resource that is likely to be critical to the performance of a data management request.

3. The computer-readable medium of claim 1 wherein identifying other data management requests in the list of data management requests comprises determining a likeness factor among the data management requests that use similar data management resources, and treating the categories as the same when the likeness factor exceeds a threshold.

4. The computer-readable medium of claim 1 wherein identifying other data management requests in the list of data management requests comprises determining whether the data management request failed because of a common resource, and when the data management request fails because of a common resource, then identifying other data management requests in the list of data management requests that request data management resources having the same category as the common resource.

5. The computer-readable medium of claim 1 wherein identifying other data management requests in the list of data management requests comprises determining whether the data management request failed because of a unique resource, and when the data management request fails because of a unique resource, then not identifying other data management requests in the list of data management requests that request data management resources having the same category as on the unique resource.

6. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by a server in a data storage management environment, cause the server to perform operations comprising:

classifying data storage requests into a plurality of categories,

wherein the data storage requests are in a queue of data storage requests to be performed in the data storage management environment,

wherein each data storage request is associated with a category from among the plurality of categories, and

wherein each data storage request requires a plurality of data storage resources to perform the respective data storage request;

checking at least one of the data storage resources required to perform a first data storage request in the queue of data storage requests to determine whether the first data storage request would fail if attempted;

if the checking indicates that the checked data storage resource would cause the first data storage request to fail if attempted, updating the queue to indicate that;

(a) the first data storage request and (b) one or more second data storage requests associated with the same category as the first data storage request should not be performed,

wherein updating the queue occurs without checking to determine whether the one or more second data storage requests would fail if attempted, and

wherein updating the queue occurs without attempting to perform the first data storage request and the one or more second data storage requests.

7. The computer-readable medium of claim 6, wherein updating the queue enables one or more third data storage requests to be performed before the first data storage request and the one or more second data storage requests, and

wherein the one or more third data storage requests are associated with a category different from the category associated with the first data storage request and the one or more second data storage requests.

8. The computer-readable medium of claim 6, wherein the category of a respective data storage request refers to a storage policy that governs the respective data storage request.

9. The computer-readable medium of claim 6, wherein the category of a respective data storage request is defined by similar data storage resource requirements.

10. The computer-readable medium of claim 6, wherein the category of a respective data storage request is defined by similar rules governing how to allocate the respective data storage resources.

11. The computer-readable medium of claim 6, wherein the instructions that, when executed by the server in a data storage management environment, cause the server to perform operations further comprising:

if the first data storage request fails in the course of being performed, further updating the queue to indicate that the one or more second data storage requests associated with the same category as the first data storage request should not be performed,

wherein the further updating of the queue occurs without checking to determine whether the one or more second data storage requests would fail if attempted, and

wherein the further updating of the queue occurs without attempting to perform the one or more second data storage requests.

12. A method for blacklisting data storage requests that are likely to fail if attempted in a data storage management environment, the method comprising:

maintaining, by a server in the data storage management environment, a list of data storage requests to be performed in the data storage management environment,

wherein each data storage request is associated with a category, and

wherein each data storage request requires multiple data storage resources to perform the respective data storage request;

performing an abbreviated resource check, by the server, to determine whether a first data storage request in the list of data storage requests would fail if attempted, wherein the abbreviated resource check comprises;

(i) determining which of the data storage resources of the multiple data storage resources needed to perform the first data storage request are physical resources as opposed to logical resources, and

(ii) performing one or more physical checks of the resources determined to be physical resources before attempting to check any of the logical resources; and

based on identifying at least one physical resource that, based on the one or more physical checks, would cause the first data storage request to fail if attempted,

identifying in the list of data storage requests, by the server, one or more second data storage requests that are associated with the same category as the first data storage request, and

removing, by the server, from the list of data storage requests;

(a) the first data storage request and (b) the one or more second data storage requests,

wherein the removing from the list occurs without attempting any further physical checks of other physical data storage resources needed to perform the first data storage request,

wherein the removing from the list occurs without attempting any checks of any logical data storage resources needed to perform the first data storage request,

wherein the removing from the list occurs without attempting any abbreviated resource check to determine whether the one or more second data storage requests would fail if attempted, and

wherein the removing from the list occurs without attempting to perform the first data storage request and the one or more second data storage requests.

13. The method of claim 12 further comprising:

classifying, by the server, the data storage requests in the list of data storage requests into multiple categories before performing any abbreviated resource checks on any data storage request in the list, including the first data storage request; and

associating, by the server, each data storage request with a respective category among the multiple categories.

14. The method of claim 12 further comprising:

performing a third data storage request before the first data storage request and the one or more second data storage requests may be attempted,

wherein the third data storage request has a lower priority in the list of data storage requests than the first data storage request and is associated with a different category than the first data storage request.

15. A method for blacklisting data storage requests that are likely to fail if attempted in a data storage management environment, the method comprising:

in reference to a queue of data storage requests to be performed in the data storage management environment,

wherein each data storage request is associated with a category from among a plurality of categories, and

wherein each data storage request requires a plurality of data storage resources to perform the respective data storage request,

performing, by a server operating in the data storage management environment, an abbreviated resource check to determine whether a first data storage request in the queue of data storage requests would fail if attempted, wherein the abbreviated resource check comprises;

(i) determining which of the data storage resources in the plurality of data storage resources needed to perform the first data storage request are physical resources and which ones are logical resources,

(ii) performing one or more physical checks of the resources determined to be physical resources before attempting to check any of the logical resources, and

(iii) if the one or more physical checks of the resources determined to be physical resources indicate that the physical resources would not cause the first data storage request to fail if attempted, performing one or more logical checks of the resources determined to be logical resources for the first data storage request;

if at least one of the physical checks and the logical checks indicate that the first data storage request would fail if attempted, removing from the queue of data storage requests;

(a) the first data storage request and (b) one or more second data storage requests associated with the same category as the first data storage request,

wherein the removing from the queue occurs without attempting any abbreviated resource checks to determine whether the one or more second data storage requests would fail if attempted, and

wherein the removing from the queue occurs without attempting to perform the first data storage request and the one or more second data storage requests.

16. The method of claim 15 further comprising:

if the first data storage request fails in the course of being performed, removing from the queue of data storage requests the one or more second data storage requests associated with the same category as the first data storage request,

wherein the removing from the queue occurs without attempting any abbreviated resource checks to determine whether the one or more second data storage requests would fail if attempted, and

wherein the removing from the queue occurs without attempting to perform the one or more second data storage requests.

17. The method of claim 15 further comprising:

classifying the data storage requests in the queue of data storage requests into the plurality of categories before performing any abbreviated resource checks on any data storage request in the queue, including the first data storage request.

18. The method of claim 15 wherein removing the first data storage request and the one or more second data storage requests from the queue results in one of:

(A) deferring the first data storage request and the one or more second data storage requests, and (B) canceling the first data storage request and the one or more second data storage requests.