DATA STORAGE RESOURCE ALLOCATION USING LOGICAL HOLDING AREAS TO RESERVE RESOURCES FOR HIGHER PRIORITY STORAGE OPERATION REQUESTS

• US 20150324226A1
• Filed: 07/21/2015
• Published: 11/12/2015
• Est. Priority Date: 06/19/2008
• Status: Abandoned Application

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv, Karandikar, Amey Vijaykumar, Wang, Yu
• Application Number:
  US14/804,726
• Publication Number:
  US 20150324226A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4818 Priority circuits therefor
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 47/805 QOS or priority aware
• CPC Class Codes: H04L 63/101 Access control lists [ACL]

Abstract

A resource allocation system begins with an ordered plan for matching requests to resources that is sorted by priority. The resource allocation system optimizes the plan by determining those requests in the plan that will fail if performed. The resource allocation system removes or defers the determined requests. In addition, when a request that is performed fails, the resource allocation system may remove requests that require similar resources from the plan. Moreover, when resources are released by a request, the resource allocation system may place the resources in a temporary holding area until the resource allocation returns to the top of the ordered plan so that lower priority requests that are lower in the plan do not take resources that are needed by waiting higher priority requests higher in the plan.

Claims

1. A system for handling a prioritized queue of data management requests, the system comprising:

a receive requests component configured to receive, into a queue, data management requests,

wherein each data management request is to perform at least one data management operation using data management resources from a pool of data management resources, and

wherein the queue includes a prioritized list of data management requests;

a handle request component configured to communicate with the receive requests component and to at least initiate each data management request,

wherein the handle request component attempts to handle each data management request in order based on the prioritized list of data management requests if data management resources from the pool of data management resources are available; and

a resource store component configured to reserve one or more data management resources released by an at least partially completed data management request, and

wherein the handle request component is configured to not provide the released one or more data management resources to a first received but lower priority data management request, but instead to provide the released one or more data management resources to a later received data management request having a higher priority than the lower priority data management request.

2. The system of claim 1, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria, and wherein at least one optimization criteria is an abbreviated check to determine if a data management request would fail, and wherein if the data management request would fail or does fail, removing the data management request from the queue of data management requests.

3. The system of claim 1, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria, and wherein at least one optimization criteria includes a blacklisted criteria that identifies at least one category of data management requests based on the data management resources used by the data management requests, and wherein if a data management request does fail or would fail, removing other data management requests having the same category from the queue of data management requests.

4. The system of claim 1, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria, and wherein at least one optimization criteria is a table of alternative resources that match data management requests to data management resources based on a user specified categorization of the data management resources.

5. The system of claim 1 wherein the resource store component returns the one or more resources to the global pool of resources after one or more lower priority data management requests are at least partially completed.

6. The system of claim 1 wherein the resource store component returns the one or more resources to the global pool of resources rather than reserving the released one or more resources if a threshold number or percentage of the data management requests in the prioritized list of data management requests have been completed.

7. The system of claim 1, further comprising:

an optimize requests component configured to communicate with the receive requests component and to optimize the queue of data management requests based on one or more optimization criteria.

8. The system of claim 1 wherein a server that operates in a data storage management environment hosts and executes one or more of:

the receive requests component, the handle request component, and the resource store component.

9. The system of claim 1 wherein the system is a resource allocation system that attempts to match data management resources in a data storage management environment to the data management requests according to the prioritized list.

10. A method for handling a queue of data storage requests to be executed in a data storage management environment, the method comprising:

at a server in the data storage management environment, providing a queue of requests,

wherein each request is to perform at least one data storage operation using data management resources from a pool of data management resources, and

wherein the requests in the queue are prioritized;

reserving, by the server, one or more data management resources released by a first request that is at least partially completed,

wherein the server refrains from providing the released one or more data management resources to a second request in the priority list, and

wherein the second request has a lower priority than a third request in the prioritized list; and

allocating, by the server, the reserved one or more data management resources to the third request, which has a higher priority in the prioritized list than the second request.

11. The method of claim 10 further comprising:

based on the reserving and the allocating, performing the third request in the data storage management environment before the second request,

thereby preserving a relative order of priority between the second request and the third request according to priorities in the queue.

12. The method of claim 10 further comprising:

after reserving the one or more data management resources released by the first request, looping back to the top of the queue of requests to handle unfulfilled requests; and

upon looping back to the top of the queue, adding the reserved one or more data management resources back to the pool of data management resources for allocation to at least one unfulfilled request, including the third request,

thereby preserving a relative order of priority between the second request and the third request according to priorities in the queue.

13. The method of claim 10 wherein the one or more data management resources are reserved to a logical holding area until the server has traversed the queue of requests according to priorities in the queue;

wherein the logical holding area comprises a plurality of data management resources reserved to the logical holding area in the course of the server traversing the queue of requests; and

after traversing the queue of requests,

(i) looping back to the top of the queue to attempt to process unfulfilled requests in the queue, and

(ii) adding the plurality of data management resources reserved to the logical holding area back to the pool of data management resources for allocation to at least one previously unfulfilled request, including the third request.

14. The method of claim 10 further comprising:

based on a measure of progress traversing the queue of requests, adding, by the server, the reserved one or more data management resources back to the pool of data management resources for allocation to the third request before the server has traversed the entire queue of requests.

15. The method of claim 10 further comprising:

based on analyzing the resource needs of the third request in the course of traversing the queue of requests, allocating the reserved one or more data management resources to the third request before fully traversing the queue of requests.

16. The method of claim 10 further comprising:

based on a measure of time that the reserved one or more data management resources have been held reserved by the server, immediately adding the reserved one or more data management resources back to the pool of data management resources before fully traversing the queue of requests.

17. The method of claim 10 further comprising:

in response to a user command, immediately adding the reserved one or more data management resources back to the pool of data management resources before fully traversing the queue of requests.

18. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by a computer, cause the computer to perform operations comprising:

receiving, by the computer operating in a data storage management environment, a queue of requests,

wherein each request is to perform at least one data storage operation using data management resources from a pool of data management resources, and

wherein the queue comprises a prioritized list of requests;

reserving one or more data management resources released by a first request that is at least partially completed,

without providing the released one or more data management resources to a second request in the priority list,

wherein the second request has a lower priority than a third request in the prioritized list;

allocating the reserved one or more data management resources to the third request, which has a higher priority in the prioritized list than the second request,

thereby preserving a relative order of priority between the second request and the third request according to the prioritized list of requests.

19. The computer-readable medium of claim 18, wherein the operations further comprise:

based on the reserving and the allocating, performing the third request in the data storage management environment before the second request according to the relative order of priority in the prioritized list.

20. The computer-readable medium of claim 18, wherein the operations further comprise:

after reserving the one or more data management resources released by the first request, looping back to the top of the queue of requests to handle unfulfilled requests; and

upon looping back to the top of the queue, adding the reserved one or more data management resources back to the pool of data management resources for allocation to at least one unfulfilled request, including the third request.

21. The computer-readable medium of claim 18, wherein the operations further comprise:

before the allocating;

(i) performing an abbreviated check of the reserved one or more data management resources to determine if the third request would fail if attempted, and

(ii) if the computer determines that the third request would fail if attempted, updating the queue of requests to indicate that the third request should not be performed.

22. The computer-readable medium of claim 18, wherein the operations further comprise:

before the allocating;

(i) performing an abbreviated check of the reserved one or more data management resources to determine if the third request would fail if attempted, and

(ii) if the computer determines that the third request would fail if attempted, updating the queue of requests to indicate that the third request should not be performed and to further indicate that other requests in the queue having a category the same as the third request also should not be performed.

23. The computer-readable medium of claim 18, wherein the operations further comprise:

if the third request fails in the course of being performed, updating the queue of requests to indicate that other requests in the queue having a category the same as the third request also should not be performed.

24. The computer-readable medium of claim 18, wherein the operations further comprise:

if the third request fails in the course of being performed, updating the queue of requests to indicate that other requests in the queue requiring at least one of the one or more data management resources as the third request also should not be performed.