Data storage resource allocation using blacklisting of data storage requests classified in the same category as a data storage request that is determined to fail if attempted

• US 10,613,942 B2
• Filed: 02/21/2017
• Issued: 04/07/2020
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv, Karandikar, Amey Vijaykumar, Wang, Yu
• Primary Examiner(s): Huisman, David J.
• Application Number:
  US15/438,433
• Publication Number:
  US 20170160971A1
• Time in Patent Office: 1,141 Days
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4818 Priority circuits therefor
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 47/805 QOS or priority aware
• CPC Class Codes: H04L 63/101 Access control lists [ACL]

Abstract

A resource allocation system begins with an ordered plan for matching requests to resources that is sorted by priority. The resource allocation system optimizes the plan by determining those requests in the plan that will fail if performed. The resource allocation system removes or defers the determined requests. In addition, when a request that is performed fails, the resource allocation system may remove requests that require similar resources from the plan. Moreover, when resources are released by a request, the resource allocation system may place the resources in a temporary holding area until the resource allocation returns to the top of the ordered plan so that lower priority requests that are lower in the plan do not take resources that are needed by waiting higher priority requests higher in the plan.

Claims

1. A method for data storage resource allocation, the method comprising:

classifying, by a server in a data management system, data storage requests into a plurality of categories,

wherein a category of a respective data storage request is based on a storage policy comprising rules for allocating data storage resources for the respective data storage request, and

wherein the data storage requests are in a queue of data storage requests to be performed in the data management system;

determining, by the server, based on one or more abbreviated checks, whether a first data storage request in the queue of data storage requests would fail if attempted, wherein the determining comprises;

(i) selecting, from a plurality of data storage resources required by the first data storage request, two or more data storage resources that are physical resources, and

(ii) performing one or more physical checks of whether the selected two or more data storage resources are available for the first data storage request, in an order based on relative chances of failure of the respective selected two or more data storage resources and ahead of any logical checks of the selected two or more data storage resources,

wherein the performing of the one or more physical checks includes determining whether hardware for performing the first data storage request is working; and

when the server determines that, based on the one or more abbreviated checks, the first data storage request would fail if attempted, updating the queue of data storage requests, by the server, to indicate that;

(a) the first data storage request and (b) one or more second data storage requests classified in the same category as the first data storage request should not be performed,

wherein updating the queue occurs without checking whether the one or more second data storage requests would fail if attempted, and

wherein updating the queue occurs without attempting to perform the first data storage request and the one or more second data storage requests.

2. The method of claim 1, wherein updating the queue enables one or more third data storage requests to be performed before the first data storage request and the one or more second data storage requests, and

wherein the one or more third data storage requests are associated with a category different from the category of the first data storage request and the one or more second data storage requests.

3. The method of claim 1, wherein a category of a given data storage request is further defined by data storage resource requirements that are similar to other data storage requests classified in a same category as the category of the given data storage request.

4. The method of claim 1, wherein a category of a given data storage request is further defined by rules for data storage resource allocation that are similar to other data storage requests classified in a same category as the category of the given data storage request.

5. The method of claim 1 further comprising:

if the first data storage request fails in the course of being performed, modifying the queue to indicate that the one or more second data storage requests classified in the same category as the first data storage request should not be performed,

wherein the modifying of the queue occurs without checking to determine whether the one or more second data storage requests would fail if attempted, and

wherein the modifying of the queue occurs without attempting to perform the one or more second data storage requests.

6. The method of claim 1, wherein the queue of data storage requests is initially ordered into an order of priority for performing data storage requests;

and

wherein updating the queue removes the first data storage request and the one or more second data storage requests from the queue, thereby changing the order of priority.

7. The method of claim 1, wherein by updating the queue, the server thereby defers performing the first data storage request and the one or more second data storage requests.

8. A data management system for data storage resource allocation based on classifying data storage requests, the system comprising:

a server hosting a storage manager comprising a database, wherein the database comprises storage policies that govern how to allocate resources for data storage requests; and

wherein the storage manager is configured to;

maintain a queue of data storage requests to be performed in the data management system,

classify data storage requests into a plurality of categories, wherein a category of a given data storage request is associated with a storage policy that governs how to allocate resources for the given data storage request, and

determine based on one or more abbreviated checks, whether a first data storage request in the queue of data storage requests would fail if attempted, comprising;

(i) selecting physical resources required by the first data storage request, and

(ii) performing one or more checks of whether the selected physical resources are available for the first data storage request, including whether hardware for performing the first data storage request is working, in an order based on relative chances of failure of the respective selected physical resources and ahead of any logical checks of the selected physical resources;

when the server determines, based on the one or more abbreviated checks, that the first data storage request would fail if attempted, updating the queue of data storage requests to indicate that;

(a) the first data storage request and (b) one or more second data storage requests classified in the same category as the first data storage request should not be performed,

wherein the server updates the queue without checking whether the one or more second data storage requests would fail if attempted, and

wherein the server updates the queue without attempting to perform the first data storage request and the one or more second data storage requests.

9. The data management system of claim 8, wherein, after the queue is updated, one or more third data storage requests are performed before the first data storage request and the one or more second data storage requests, and

wherein the one or more third data storage requests are associated with a category different from the category of the first data storage request and the one or more second data storage requests.

10. The data management system of claim 8, wherein a category of a given data storage request is further defined by data storage resource requirements that are similar to other data storage requests classified in the same category as the category of the given data storage request.

11. The data management system of claim 8, wherein a category of a given data storage request is further defined by rules for data storage resource allocation that are similar to other data storage requests classified in the same category as the category of the given data storage request.

12. The data management system of claim 8, wherein the server is further configured to:

if the first data storage request fails in the course of being performed, modify the queue to indicate that the one or more second data storage requests classified in the same category as the first data storage request should not be performed,

wherein the server modifies the queue without checking to determine whether the one or more second data storage requests would fail if attempted, and

wherein the server modifies the queue without attempting to perform the one or more second data storage requests.

13. The data management system of claim 8, wherein by updating the queue, the server thereby defers performing the first data storage request and the one or more second data storage requests.