LOAD BALANCING ACROSS MULTIPLE DATA PATHS

• US 20180011767A1
• Filed: 08/21/2017
• Published: 01/11/2018
• Est. Priority Date: 05/09/2014
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kochunni, Jaidev O., Liu, Chong, Vijayan, Manoj Kumar, Kottomtharayil, Rajiv
• Granted Patent: US 10,310,950 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 11/349 for interfaces, buses
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.

Claims

1. An apparatus for backing up data from a primary storage to a secondary storage, the apparatus comprising:

at least one processor;

a module that identifies a first subset of the data to be backed up, and a second subset of the data to be backed up, both from the primary storage,

wherein the first and second subsets of the data are backed up based on a value of an application-level characteristic that is different between the first subset of the data and the second subset of the data;

a module that allocates a job for transferring the data from the primary storage to the secondary storage, wherein the secondary storage comprises multiple storage locations;

a module that divides the job into multiple data streams, wherein each stream has a corresponding data path for carrying data, and

wherein each data path is associated with a corresponding and different media agent and a corresponding and different destination storage location from the multiple storage locations of the secondary storage; and

a module that stores, for each unit of data transferred from the primary storage to the secondary storage, information identifying a primary storage location for the unit of data and a destination storage location at which the unit of data is backed up;

wherein the first subset of data and the second subset of data are backed up using at least two different streams.

2. The apparatus of claim 1, further including:

a module that receives, from the corresponding media agent, the information identifying the primary storage location and the corresponding destination storage location; and

a module that stores the information in an index table.

3. The apparatus of claim 1, further including:

a module that receives, for the job, a storage policy that specifies at least one rule for assigning a media agent and the corresponding and different destination storage location; and

a module that allocates resources to the job according to the storage policy.

4. The apparatus of claim 3, wherein the storage policy further specifies a de-duplication rule for the job, wherein the rule specifies at least one of a directory to be used for de-duplication and a hash function to be used for de-duplication.

5. The apparatus of claim 1, further comprising:

a module that maintains, for the primary storage location, information identifying the first subset of data and a first group of destination storage locations at which units making up the first subset of data are stored, and the second subset of data and a second group of destination storage locations at which units making up the second subset of data are stored.

6. The apparatus of claim 1, wherein the at least two different streams comprise a first stream configured to transfer data over a first host bus adapter and a second stream configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.

7. The apparatus of claim 1, wherein the first subset of the data includes electronic mail items and the second subset of data includes journaling data for the electronic mail items.

8. The apparatus of claim 1, wherein the first subset of the data includes database entries and the second subset of data includes log tables for the database.

9. A method of backing up data from a primary storage to a secondary storage, the method comprising:

identifying a first subset of the data to be backed up and a second subset of the data to be backed up from the primary storage based on a value of an application level characteristic that is different for the first subset of the data and the second subset of the data;

allocating a job for transferring the data from the primary storage to the secondary storage, wherein the secondary storage comprises a plurality of storage locations;

dividing the job into multiple streams, each stream having a corresponding data path for carriage of data, wherein each data path is associated with a corresponding media agent and a corresponding destination storage location from the plurality of storage locations of the secondary storage;

maintaining, for each unit of data transferred from the primary storage to the secondary storage, information identifying a primary storage location for the unit of data and a destination storage location at which the unit of data is backed up;

wherein the first subset of data and the second subset of data are backed up using at least two different streams.

10. The method of claim 9, further including:

receiving, from the corresponding media agent, the information identifying the primary storage location and the corresponding destination storage location.

11. The method of claim 9, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and the corresponding destination storage location; and

allocating resources to the job according to the storage policy.

12. The method of claim 11, wherein the storage policy further specifies a de-duplication rule for the job.

13. The method of claim 9, further comprising:

maintaining, for the primary storage location, information identifying the first subset of data and a first group of destination storage locations at which units making up the first subset of data are stored, and the second subset of data and a second group of destination storage locations at which units making up the second subset of data are stored.

14. The method of claim 9, wherein the at least two different streams comprise a first stream configured to transfer data over a first host bus adapter and a second stream configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.

15. The method of claim 9, wherein the primary storage location comprises a source file name.

16. A method of restoring data from a secondary storage to a primary storage, the data having been copied from the primary storage to the secondary storage in multiple subsets based on an application-level characteristic of each subset, the method comprising:

allocating a job for transferring the data from the secondary storage to the primary storage, wherein the secondary storage comprises a plurality of storage locations, and wherein each storage location stores at least one of the multiple subsets of the data;

determining, for each unit of data to be transferred from the secondary storage to the primary storage, information identifying a primary storage location for the unit of data to which the unit of data is to be restored;

dividing the job into multiple streams, wherein each stream has a corresponding data path for carrying data, and wherein each data path is associated with a corresponding and different media agent and a corresponding and different storage location from the plurality of storage locations of the secondary storage;

assigning the data path for restoring one of the subset of the data stored at the corresponding storage location; and

restoring the multiple subsets of data wherein at least two subsets of data are restored using two different data paths.

17. The method of claim 16, further including:

providing, to the corresponding media agent, the information identifying the primary storage location and the corresponding destination storage location.

18. The method of claim 16, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and the corresponding destination storage location; and

allocating resources to the job according to the storage policy.

19. The method of claim 18, wherein the storage policy further specifies a de-duplication rule for the job, wherein the rule specifies at least one of a directory to be used for de-duplication and a hash function to be used for de-duplication.

20. The method of claim 16, wherein the at least two different streams comprise a first stream configured to transfer data over a first host bus adapter and a second stream configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.