Systems and methods for granular resource management in a storage network

• US 9,313,143 B2
• Filed: 09/04/2013
• Issued: 04/12/2016
• Est. Priority Date: 12/19/2005
• Status: Expired due to Fees

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kottomtharayil, Rajiv, Gokhale, Parag, Muller, Marcus S.
• Primary Examiner(s): TSAI, SHENG JEN
• Application Number:
  US14/018,014
• Publication Number:
  US 20140012994A1
• Time in Patent Office: 951 Days
• Field of Search: 711/154
• US Class Current: 1/1
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

In accordance with some aspects of the present invention, systems and methods are provided for dynamically and/or automatically selecting and/or modifying data path definitions that are used in performing storage operations on data. Alternate data paths may be specified or selected that use some or all resources that communicate with a particular destination to improve system reliability and performance. The system may also dynamically monitor and choose data path definitions to optimize system performance, conserve storage media and promote balanced load distribution.

Claims

1. A method for dynamically routing data to a plurality of storage operation paths in a data storage system, the method comprising:

accessing a first storage policy of a plurality of storage policies, the first storage policy comprising at least a first storage operation path that associates at least a first media agent with a first storage device in a first static relationship;

accessing a second storage policy of the plurality of storage policies, the second storage policy comprising at least a second storage operation path that associates a second media agent with the first storage device in a second static relationship, wherein the second media agent conducts data to the first storage device via the second storage operation path;

automatically evaluating with one or more computer hardware processors, the first and second storage polices and the first and second static relationships to determine that both the first and second storage polices use both the first and second storage operation paths to conduct data to at least the first storage device;

reducing a number of the plurality of policies by automatically combining the first and second storage policies into one comprehensive storage policy such that the first and second storage polices no longer exist as separate storage policies, wherein the comprehensive storage policy associates at least the first and second storage operation paths to the first storage device in a dynamic relationship, wherein both the first and second storage paths of the comprehensive storage policy conduct data to at least the first storage device;

evaluating network operating conditions;

automatically adding at least a third storage operation path to the comprehensive storage policy based on network congestion that is predicted to occur, wherein the third storage operation path is an alternate data path that is different than the first and second storage operation paths;

dynamically allocating the third storage operation path in the comprehensive storage policy to conduct data to the first storage device prior to the network congestion that is predicted to occur;

dynamically allocating with one or more computer hardware processors, at least one of the first and second storage operation paths in the comprehensive storage policy to conduct data to the first storage device when the network congestion is not predicted to occur.

2. The method of claim 1 wherein the network operating conditions include anyone of the following characteristics:

data transfer rate, load balancing, resource exhaustion, transmission congestion, or performance optimization.

3. The method of claim 1 wherein dynamically allocating the first and second storage paths is based on user preference.

4. The method of claim 1 wherein information about the first and second storage operation paths is obtained from an index cache.

5. The method of claim 1 wherein information about the first and second storage operation paths is obtained from a metabase.

6. The method of claim 1 wherein evaluating network operation comprises evaluating at least one of the group comprising:

origination point, destination point, transmission resources scheduled to be involved, and a process-based netlist.

7. The method of claim 1 wherein evaluating network operation comprises forecasting network usage.

8. The method of claim 1 wherein evaluating network operation comprises determining which storage operation paths are likely to experience an adverse impact due to changing conditions.

9. The method of claim 1 wherein dynamically allocating comprises maintaining a substantially constant data load.

10. The method of claim 1 wherein dynamically allocating comprises preventing a failover condition.

11. A storage operation system comprising:

a plurality of storage devices;

a plurality of storage policies;

a first media agent executing in one or more computer hardware processors, the first media agent configured to manage the flow of data to at least a first storage device based on a first storage policy of the plurality of storage polices, wherein the first storage policy defines a first storage operation path that associates the first media agent and the first storage device in a first static relationship;

a second media agent executing in one or more computer hardware processors, the second media agent configured to manage the flow of data to the first storage device based on a second storage policy of the plurality of storage polices, wherein the second storage policy defines a second storage operation path that associates the second media agent and the first storage device in a second static relationship; and

a management component executing in one or more computer hardware processors, the management component configured to;

automatically evaluate the first and second storage policies and the first and second static relationships to determine that both the first and second storage polices use both the first and second storage paths to conduct data to at least the first storage device;

reduce a number of the plurality of storage policies by automatically combining the first and second storage policies into one comprehensive storage policy such that the first and second storage polices no longer exist as separate storage policies, wherein the comprehensive storage policy associates at least the first and second storage paths to the first storage device in a dynamic relationship, wherein both the first and second data paths of the comprehensive storage policy conduct data to at least the first storage device;

evaluate network operating conditions, and

automatically add at least a third storage operation path to the comprehensive storage policy based on network congestion that is predicted to occur, wherein the third data path is an alternate data path this is different than the first and second storage operation paths;

dynamically allocating the third storage operation path in the comprehensive storage policy to conduct data to the first storage device prior to the network congestion that is predicted to occur; and

dynamically allocate at least one of the first and second storage paths in the comprehensive storage policy to conduct data to the first storage device based when the network congestion is not predicted to occur.

12. The system of claim 11 wherein the network operating conditions include anyone of the following characteristics:

data transfer rate, load balancing, resource exhaustion, transmission congestion, or performance optimization.

13. The system of claim 11 wherein the management component is configured to dynamically allocate the first and second storage paths based on user preference.

14. The system of claim 11 wherein the management component is configured to obtain information about the first and second storage operation paths from an index cache.

15. The system of claim 11 wherein the management component is configured to obtain information about the first and second storage operation paths from a metabase.

16. The system of claim 11 wherein the management component is configured to evaluate network operation based on at least one of the group comprising:

origination point, destination point, transmission resources scheduled to be involved, and a process-based netlist.

17. The system of claim 11 wherein the management component is configured to evaluate network operation based on a forecast of network usage.

18. The system of claim 11 wherein the management component is configured to evaluate network operation based on which storage operation paths are likely to experience an adverse impact due to changing conditions.

19. The system of claim 11 wherein the management component is configured to dynamically allocate the first and second storage operation paths based on maintaining a substantially constant data load.

20. The system of claim 11 wherein the management component is configured to dynamically allocate the first and second storage operation paths based on preventing a failover condition.