Systems and methods of hierarchical storage management, such as global management of storage operations

• US 7,831,566 B2
• Filed: 10/31/2007
• Issued: 11/09/2010
• Est. Priority Date: 12/22/2006
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kottomtharayil, Rajiv, Kavuri, Srinivas
• Primary Examiner(s): Fleurantin; Jean B.
• Assistant Examiner(s): Hershley; Mark E
• Application Number:
  US11/982,324
• Publication Number:
  US 20080228785A1
• Time in Patent Office: 1,105 Days
• Field of Search: 707/200, 707/609, 707/640, 707/651, 707/654, 707/687, 707/694, 711/161, 711/162, 711/167
• US Class Current: 707/651
• CPC Class Codes: G06F 16/10 File systems; File servers

Abstract

A system and method for setting global actions in a data storage system is described. In some examples, the system determines a policy based on information from the system, and implements that policy to the system. In some examples, the system adds or modifies global filters based on information from the system.

Claims

1. A method of adjusting future filter policies in a data storage system managed by a global storage management server, the method comprising:

sampling, prior to a time T0, at least some of the data storage resources performing a certain storage operation, wherein the sampling determines data transfer loads on the data storage resources;

determining, at time T0, a job usage factor for each of the data storage resources using the determined data transfer loads, wherein the job usage factor relates to storage performance of a data storage resource during the certain storage operation;

identifying, at a time T1 later than T0;

a future storage operation currently associated with a first filter policy; and

a second, different filter policy;

wherein the future storage operation is to be performed under the first filter policy and at a time that is later than T1; and

wherein, the first filter policy and the second filter policy each define at least two of the following;

when to perform data storage operations, how often to perform data storage operations, a data storage location to use during a data storage operation, a relationship between components to use during a data storage operation, accessible datapipes for use during data storage operations, data storage retention schemes, data compression or encryption requirements, and a network pathway to use for a storage operation; and

,

wherein the identifying is performed at least in part by analyzing at least two iob usage factors in order to determine that performing the future storage operation in accordance with the first filter policy would result in a higher mean number of jobs in the data storage system than if the future storage operation were performed in accordance with the second, different filter policy;

associating the identified future storage operation with the second, different filter policy instead of the first filter policy; and

performing, at a time T2 that is subsequent to T1, the identified storage operation in accordance with the second filter policy.

2. The method of claim 1, wherein determining the job usage factor for each of the data storage resources comprises determining a mean number of jobs transferred during the certain storage operation.

3. The method of claim 1, wherein determining the job usage factor for each of the data storage resources comprises determining a median number of jobs transferred during the certain storage operation.

4. The method of claim 1, further comprising:

determining a job delay factor for each of the data storage resources, wherein the job delay factor relates to a number of data transfer jobs waiting at a data storage resource during the certain storage operation; and

wherein

the identifying further comprises analyzing at least two job delay factors to determine that performing the future storage operation in accordance with the first filter policy would result in a higher mean number of lobs in the data storage system than if the future storage operation were performed in accordance with the second, different filter policy.

5. A system for adjusting the transfer of data from a source to a destination, the system comprising:

a server connected to two or more storage modules actively transferring data from the source to the destination in accordance with a first storage policy, wherein the server;

generates a report related to data transfer performance of the two or more storage modules, wherein the report indicates results of a quantitative comparison of data transfer performances between the two or more storage modules, wherein the quantitative comparison reflects an excess number of storage jobs waiting for resources at a given time;

utilizes the report to identify a redistribution of the active transfer of data across resources of

the two or more storage modules after the given time; and

a data transfer modification component, wherein the data transfer modification component modifies the active transfer of data from the source to the destination so that the transfer is no longer performed in accordance with the first storage policy but is instead performed in accordance with the identified redistribution after the given time,

wherein, the first storage policy defines at least two of the following;

when to perform storage operations,

how often to perform storage operations,

a storage location to use during a storage operation,

a relationship between components to use during a storage operation,

accessible datapipes usable during storage operation,

storage retention schemes,

compression or encryption requirements, and

a network pathway to use for a storage operation.

6. The system of claim 5, wherein the report comprises results of a quantitative comparison of job usage between the two or more storage modules.

7. The system of claim 5, wherein the report comprises results of a quantitative comparison of job delay between the two or more storage modules.

8. The system of claim 5, wherein the quantitative comparison relates to queuing theory.

9. The system of claim 5, wherein the quantitative comparison relates to the M/M/1 model of queuing theory.

10. The system of claim 5, wherein the quantitative comparison of data transfer performances between the two or more storage modules compares a number of jobs at each storage module with a number of storage modules.

11. A system for defining a filter policy in a hierarchical data storage system, the system comprising:

two or more data storage resources, wherein the data storage resources comprise;

a first storage cell that performs a first number of data storage jobs during a data storage operation, wherein a storage cell is configured to transfer data from a source to a destination; and

a second storage cell, different from the first storage cell, that performs a second number of data storage jobs during the data storage operation;

wherein the first storage cell and the second storage cell each individually comprise a storage manager and comprise;

a primary storage database,

a media agent and a secondary database, or

a primary storage database, a media agent and a secondary database; and

a server configured to assign a filter policy to the first storage cell and the second storage cell, wherein the server comprises;

a comparison component that compares the performed first number of jobs to the performed second number of jobs; and

a filter assignment component that assigns the filter policy to the first storage cell and the second storage cell based on the comparison;

wherein the filter policy defines at least two of the following;

when to perform storage operations,

how often to perform storage operations,

a storage location to use during a storage operation,

a relationship between components to use during a storage operation,

accessible datapipes for storage operations,

retention schemes for stored data,

data compression or encryption requirements, and

a network pathway to use for a storage operation.

12. The system of claim 11, wherein assigning the filter policy includes assigning a global filter policy that applies to all storage cells within the hierarchical data storage system.

13. The system of claim 11, wherein assigning the filter policy includes assigning a first filter policy that applies to the first storage cell and assigning a second filter policy at least partially different than the first filter policy to the second storage cell.

14. The system of claim 11, further comprising:

an interface component, wherein the interface component presents the assigned filter policy to an administrator of the hierarchical data storage system.