Data storage resource allocation by employing dynamic methods and blacklisting resource request pools

• US 8,352,954 B2
• Filed: 06/19/2008
• Issued: 01/08/2013
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv, Karandikar, Amey, Wang, Yu
• Primary Examiner(s): An, Meng
• Assistant Examiner(s): Rashid, Wissam
• Application Number:
  US12/142,445
• Publication Number:
  US 20090320037A1
• Time in Patent Office: 1,664 Days
• Field of Search: 711/154, 718/105
• US Class Current: 718/105
• CPC Class Codes: G06F 13/16 for access to memory bus G06F13/28 takes precedence
• CPC Class Codes: G06F 2209/502 Proximity
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/5016 the resource being the memory

Abstract

A resource allocation system begins with an ordered plan for matching requests to resources that is sorted by priority. The resource allocation system optimizes the plan by determining those requests in the plan that will fail if performed. The resource allocation system removes or defers the determined requests. In addition, when a request that is performed fails, the resource allocation system may remove requests that require similar resources from the plan. Moreover, when resources are released by a request, the resource allocation system may place the resources in a temporary holding area until the resource allocation returns to the top of the ordered plan so that lower priority requests that are lower in the plan do not take resources that are needed by waiting higher priority requests higher in the plan.

Claims

1. A method of assigning data storage requests to data storage resources, the method comprising:

receiving at least two data storage requests to be performed in multiple locations,

wherein each of the multiple locations includes at least one separate data storage resource of a data storage system;

determining, for each received data storage request, whether or not a requested resource has been blacklisted by comparing the received data storage request to a generated blacklist table to blacklist that request based on a resource requirement category of that request;

for each data storage request, and before receiving data storage requests to be executed, determining at least first and second sets of data storage resources to handle the request,

wherein the first set of data storage resources is a preferred set of data storage resources to handle the request, while the second set of data storage resources is an alternate set of data storage resources to handle the request, and

wherein the first data storage resources differs from the second set of data storage resources;

before receiving data storage requests to be executed, establishing a data storage resource allocation based at least in part on one of the at least first and second sets of data storage resources determined to handle each data storage request

wherein establishing the data storage resource allocation includes building a grid allocation table,

wherein columns of the grid allocation table include resources with common characteristics, and rows of the grid allocation table include the data storage requests, or

wherein rows of the grid allocation table include resources with common characteristics, and columns of the grid allocation table include the data storage requests;

determining whether requested resources from a failed request comprise common or unique resources; and

selectively adding the failed request to the blacklist table when the failed request corresponded to a common resource, and not adding the failed request to the blacklist table when the failed request corresponded to a unique requested resource.

2. The method of claim 1 wherein the preferred set of data storage resources is determined based on geographic proximity of one or more data objects related to a data storage request and available data storage resources.

3. The method of claim 1 wherein the preferred set of data storage resources is determined based on network proximity of one or more data objects related to a data storage request and available data storage resources.

4. The method of claim 1 wherein the wherein the second set of data storage resources are employed when at least one resource in the first set of data storage resources is unavailable to handle the request.

5. The method of claim 1 wherein determining at least the second set of data storage resources to handle the request includes evaluating one or more rules to determine an alternate data path for performing a data storage request, wherein at least one rule is an overflow rule that indicates an alternate data path to use for performing the data storage request when the preferred set of data storage resources is unavailable.

6. The method of claim 1 wherein determining at least the second set of data storage resources to handle the request includes evaluating one or more rules to determine an alternate data path for performing a data storage request, wherein at least one rule is an overflow rule that distributes data storage requests across sets of data storage resources in an alternating fashion to evenly distribute the data storage requests.

20. The method of claim 1, wherein the resource requirement category comprises at least an access to a tape drive, an allocation of a stream for that tape drive, an available tape on which to store data, and a media agent computer to process and monitor the request.

7. A system having a processor for dividing data management requests based on data management resources available for performing the data management requests, the system comprising:

a means for receiving multiple data management requests,

wherein the data management requests specify one or more resource requirements;

a means for determining, for each received data management request, whether or not a requested resource has been blacklisted by comparing the received data storage request to a generated blacklist table to blacklist that request based on a resource requirement category of that request;

a means for building a resource allocation grid by;

grouping resources in columns according to common characteristics and grouping the received data management requests in rows of the resource allocation grid, or

grouping resources in rows according to common characteristics and grouping the received data management requests in columns of the resource allocation grid;

a means for allocating the multiple data management requests to the data management resources available for performing the data management by optimizing of resources within the resource allocation grid; and

a means for handling the data management requests according to the resource allocation grid;

a means for determining whether requested resources from a failed request comprise common or unique resources; and

a means for selectively adding the failed request to the blacklist table when the failed request failed because of requested common resources and not adding the failed request to the blacklist table when the failed request failed because of unique requested resources;

wherein at least one of the means for receiving, the means for determining, the means for building and the means for handling comprises a hardware module.

8. The system of claim 7 further comprising a means for modifying the resource allocation grid to include alternate data paths for performing the data management requests when the allocated data management resources are unavailable.

9. The system of claim 7 wherein the means for building a resource allocation grid determines the dimensions of the grid based on a categorization of data management resources based on geographical location of the data management resources.

10. The system of claim 7 wherein the means for building a resource allocation grid determines the dimensions of the grid based on a categorization of data management resources based on network location of the data management resources.

11. The system of claim 7 wherein the means for building a resource allocation grid determines the dimensions of the grid based on a categorization of data management resources based on physical location of the data management resources.

12. The system of claim 7 wherein the means for building a resource allocation grid determines the dimensions of the grid based on a categorization of data management resources based on logical location of the data management resources.

13. The system of claim 7 wherein the resource allocation grid facilitates performing the data management requests within a selected period of time.

21. The system of claim 7, wherein the resource requirement category comprises at least an access to a tape drive, an allocation of a stream for that tape drive, an available tape on which to store data, and a media agent computer to process and monitor the request.

14. A non-transitory computer-readable medium encoded with instructions for controlling a computer system in performing data management operations on data objects located at different physical locations, by a method comprising:

receiving a list of multiple data management operations;

receiving a list of multiple data management resources for use in performing the multiple data management operations;

determining, for each data management operation, whether or not a requested resource has been blacklisted by comparing the received data storage request to a generated blacklist table to blacklist that operation based on a resource requirement category of that operation;

for at least some of the multiple data management resources;

determining characteristics of the data management resource, and

grouping at least two of the data management resources based on a similarity of at least one determined characteristic;

for each data management operation in the list of multiple data management operations, determining a location of one or more data objects affected by the data management operation;

creating a logical table for allocating the data management resources,

wherein the created logical table for resource allocation selects one or more data management resources to perform each of the multiple data management operations based at least in part on the determined physical location of the one or more data objects affected by the data management operation, and on the grouping of at least two of the data management resources based on a similarity of determined characteristics;

performing the data management operations based on the created logical table for resource allocation, wherein performing the data management operations comprises performing in parallel operations along one dimension of the created logical table;

determining whether requested resources from a failed request comprise common or unique resources; and

selectively adding the failed request to the blacklist table when the failed request failed because of requested common resources and not adding the failed request to the blacklist table when the failed request failed because of unique requested resources.

15. The computer-readable medium of claim 14 wherein the at least one determined characteristic comprises a geographic criteria based on subsets of data identified by a user and related to office locations of a business organization.

16. The computer-readable medium of claim 14 wherein grouping at least two of the data management resources based on a similarity of at least one determined characteristic comprises grouping the similar data management resources within a column of a table.

17. The computer-readable medium of claim 14 wherein performing the data management operations comprises performing in priority order operations along one dimension of the created logical table.

18. The computer-readable medium of claim 14 wherein determining the location of the data objects comprises determining a geographic location of a computer on which the data objects are stored.

19. The computer-readable medium of claim 14 wherein determining the location of the data objects comprises determining a network location of the data objects.

22. The computer-readable medium of claim 14, wherein the resource requirement category comprises at least an access to a tape drive, an allocation of a stream for that tape drive, an available tape on which to store data, and a media agent computer to process and monitor the request.