System and method for performing multi-path storage operations

• US 7,620,710 B2
• Filed: 12/19/2005
• Issued: 11/17/2009
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Chen, Ho-Chi, Kottomtharayil, Rajiv, Retnamma, Manoj Vijayan
• Primary Examiner(s): Pwu; Jeffrey
• Assistant Examiner(s): Nguyen; Van Kim T
• Application Number:
  US11/313,224
• Publication Number:
  US 20070143497A1
• Time in Patent Office: 1,429 Days
• Field of Search: 709/217, 709/218, 709/219, 709/223, 709/224, 709/226, 707/10, 707/101, 707/205, 711/111, 711/156, 711/157, 711/165, 711/166
• US Class Current: 709/223
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Systems and methods for allocating transmission resources within a computer network are provided. In some embodiments of the invention, communication links may be assigned based on predefined preferences or system configuration to facilitate the transfer of data from one point in the network to another. In other embodiments, system operation may be monitored and communication paths be assigned dynamically based on this information to improve system operation and provide improved failover response, load balancing and to promote robust data access via alternative routes.

Claims

1. A method for allocating data in a storage operation, the method comprising:

receiving a storage policy that defines the migration of data in a multi-tier data storage system, wherein the storage policy comprises at least one storage operation for automatically migrating storage data from a first storage device to a second storage device based on retention criteria;

in response to the storage policy, determining a size of the storage data to be moved to the second storage location in the storage operation;

in response to a static mode of operation, polling the availability of a plurality of HBA communication paths assigned to the first storage device and if available, transferring the storage data to the second storage location via the HBA communications;

in response to a dynamic mode of operation, determining whether the second storage device has adequate storage resources to store the storage data;

if the storage resources in the second storage device are adequate to store a portion of the storage data, but not all of the storage data, selecting a third storage device;

dynamically selecting at least first and second HBA communication paths to transfer the primary storage data to the second and third storage devices;

receiving a threshold transmission capacity of each of the first and second HBA communication paths;

parsing the storage data according to the threshold transmission capacity of the first and second HBA communication paths and wherein the parsing is further based on the storage resources available in the second and third storage devices, wherein the parsing creates at least a first set of parsed data units and a second set of parsed data units;

transferring the first set of parsed data units via the first HBA communication path to the second storage device and transferring the second set of parsed data units via the second HBA communication path to the third storage device.

2. The method of claim 1, further comprising selecting a media module from a plurality of media modules in communication with the at least one storage device.

3. The method of claim 1, wherein the at least one storage device comprises a single storage device.

4. The method of claim 1, wherein said parsing based on said threshold transmission capacity is performed by the media module.

5. The method of claim 1, further comprising an availability of each selected HBA communication path.

6. The method of claim 1, further comprising monitoring an available storage capacity of the at least one storage device.

7. The method of claim 6, additionally comprising moving data between a media module and the second storage device only after determining, based on said monitoring, that the first storage device no longer has available storage capacity.

8. A system for allocating data in a storage operation, the system comprising:

a plurality of storage devices;

a media module configured to transfer data from a client device to the plurality of storage devices;

a plurality of host bus adapter (HBA) devices in communication the media module and in communication with the plurality of storage devices to form a plurality of communication paths each comprising the media module, one of the plurality of HBA devices and one of the plurality of storage devices, wherein the media module is configured to communicate with each of the plurality of storage devices through each of the plurality of HBA devices;

a management module configured to store at least one storage policy comprising storage criteria for performing a storage operation on the data wherein the storage policy defines the migration of data in a multi-tier data storage system, and wherein the storage policy comprises at least one storage operation for automatically transferring data from at least a first storage device to at least a second storage device, the storage criteria comprising at least a threshold capacity associated with each of the plurality of communication paths; and

at least one data agent operating on the client device and configured to parse the data when a size of the data exceeds the threshold capacity of at least one of the plurality of communication paths,

wherein the media module is further configured to select, based on network operating conditions, which of the plurality of communication paths to use for performing the storage operation on the data, and wherein the media module is further configured to determine whether the second storage device has adequate storage resources to store the data and if the storage resources in the second storage device are adequate to store a portion of the data, but not all of the data, the media module is configured to dynamically select a third storage device; and

wherein the data agent parses the data based on the threshold transmission capacity of the communication paths and based on the storage resources available in the second and third storage devices.

9. The system of claim 8, wherein the at least one storage device comprises a single storage device, and wherein the media module communicates with the single storage device through the selected communication paths.

10. The system of claim 8, further comprising a first index maintained by the management module, wherein the first index associates the media module with the at least one storage device.

11. The system of claim 10, further comprising a second index maintained by the media module, wherein the second index indicates a particular location of the data on the at least one storage device, the information in the second index being different than the information in the first index.

12. The system of claim 8, wherein the management module is further configured to store configuration information assigning the plurality of HBA devices to the media module.

13. The system of claim 8, wherein the management module is configured to dynamically assign which of the plurality of HBA devices can be used by the media module to perform the storage operation.

14. The system of claim 8, wherein the media module comprises a first media module, and wherein the system further comprises a plurality of media modules including the first media module, each of the plurality of media modules being configured to communicate with each of the plurality of storage devices.

15. The system of claim 14, wherein the management module is further configured to select the first media module from the plurality of media modules to perform the storage operation.

16. The system of claim 15, wherein the management module is further configured to select the at least one storage device for performing the storage operation.

17. The system of claim 16, wherein at least one of the management module and the media module is configured to monitor an availability of the selected communication paths during performance of the storage operation.

18. The system of claim 17, wherein the media module is further configured to switch from using a first communication path to using a second communication path when the first communication path is determined to become unavailable during performance of the storage operation.

19. A system for allocating data in a storage operation, the system comprising:

means for storing data comprising at least a first storage device, a second storage device and a third storage device;

means for transferring client data to said storing means, wherein said transferring means communicates with each of said storing means through each of a plurality of HBA devices, each of the plurality of host bus adapter (HBA) devices being coupled between said transferring means and said storing means to form a plurality of communication paths each comprising said transferring means, one of the plurality of HBA devices and one of said storing means;

means for managing at least one storage policy that defines the migration of data in a multi-tier data storage system, wherein the storage policy comprises at least one storage operation for automatically migrating storage data from at least a first storage device to at least a second storage device based on retention criteria, wherein the storage policy further comprises storage criteria for performing a storage operation on the client data, the storage criteria comprising at least a threshold capacity associated with each of the plurality of communication paths; and

means for operating on the client data to parse the client data when a size of the client data exceeds the threshold capacity of at least one of the plurality of communication paths,

wherein said transferring means is further configured to select, based on network operating conditions, which of the plurality of communication paths to use for performing the storage operation on the client data, and wherein the transferring means is further configured to determine whether the second storage device has adequate storage resources to store the data and if the storage resources in the second storage device are adequate to store a portion of the data, but not all of the data, the media module is configured to dynamically select the third storage device; and

wherein the data agent parses the data based on the threshold transmission capacity of the communication paths and based on the storage resources available in the second and third storage devices.

20. The system of claim 19, wherein said managing means is further configured to dynamically assign which of the plurality of HBA devices can be used by said transferring means to perform the storage operation.