Load balancing across multiple data paths

• US 9,740,574 B2
• Filed: 05/09/2014
• Issued: 08/22/2017
• Est. Priority Date: 05/09/2014
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kochunni, Jaidev O., Liu, Chong, Vijayan, Manoj Kumar, Kottomtharayil, Rajiv
• Primary Examiner(s): Gillis, Brian J
• Assistant Examiner(s): Ling, Amy
• Application Number:
  US14/274,622
• Publication Number:
  US 20150324255A1
• Time in Patent Office: 1,201 Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 11/349 for interfaces, buses
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.

Claims

1. An apparatus for backing up data from a primary storage to a secondary storage, the apparatus comprising:

at least one processor;

a module identifies data to be backed up from primary storage, including;

a first subset of the data to be backed up, which is tagged as an application-level characteristic of user data, and

a second subset of the data to be backed up, which is tagged as an application-level characteristics of index data or log data;

a module allocates a job for transferring the data from the primary storage to the secondary storage,

wherein the secondary storage comprises multiple storage locations;

a module divides the job into multiple data streams,

wherein each data stream of the multiple data streams has a corresponding data path for carrying data, and is associated with the first subset of the data or the second subset of the data based on the identified application-level characteristics tagged to the data, and wherein each data path is associated with a corresponding and different media agent and a corresponding and different destination storage location from the multiple storage locations of the secondary storage; and

a module stores, for each unit of data transferred from the primary storage to the secondary storage, information identifying a primary storage location for the unit of data and a destination storage location at which the unit of data is backed up; and

a module transfers the data to be backed up from the primary storage to the secondary storage,

wherein the first subset of the data is transferred via first data streams of the multiple data streams that are associated with the first subset of the data based on the identified application-level characteristics of the data, and

wherein the second subset of the data is transferred via second, different, data streams of the multiple data streams that are associated with the second subset of the data based on the identified application-level characteristics of the data.

2. The apparatus of claim 1, further including:

a module receives, from the corresponding media agent, the information identifying the primary storage location and the corresponding destination storage location; and

a module stores the information in an index table.

3. The apparatus of claim 1, further including:

a module receives, for the job, a storage policy that specifies at least one rule for assigning a media agent and the corresponding and different destination storage location; and

a module allocates resources to the job according to the storage policy.

4. The apparatus of claim 3, wherein the storage policy further specifies a de-duplication rule for the job, wherein the rule specifies at least one of a directory to be used for de-duplication and a hash function to be used for de-duplication.

5. The apparatus of claim 1, further comprising:

a module maintains, for the primary storage location, information identifying the first subset of data and a first group of destination storage locations at which units making up the first subset of data are stored, and the second subset of data and a second group of destination storage locations at which units making up the second subset of data are stored.

6. The apparatus of claim 1, wherein the at least two different streams comprise the first data streams configured to transfer data over a first host bus adapter and the second data streams configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.

7. The apparatus of claim 1, wherein the first subset of the data includes electronic mail items and the second subset of data includes journaling data for the electronic mail items.

8. The apparatus of claim 1, wherein the first subset of the data includes database entries and the second subset of data includes log tables for the database.

9. A method of backing up data from a primary storage to a secondary storage, the method comprising:

identifying a first subset of the data to be backed up and a second subset of the data to be backed up from the primary storage based on a value of an application level characteristic that is different for the first subset of the data and the second subset of the data,

wherein the first subset of the data includes data tagged as user data, and

wherein the second subset of the data includes data tagged as control data, index data or log data;

allocating a job for transferring the data from the primary storage to the secondary storage, wherein the secondary storage comprises a plurality of storage locations;

dividing the job into multiple streams, each stream having a corresponding data path for carriage of data, wherein each data path is associated with a corresponding media agent and a corresponding destination storage location from the plurality of storage locations of the secondary storage;

maintaining, for each unit of data transferred from the primary storage to the secondary storage, information identifying a primary storage location for the unit of data and a destination storage location at which the unit of data is backed up; and

transferring the data to be backed up using one or more streams for the first subset of data and one or more different streams for the second subset of data.

10. The method of claim 9, further including:

receiving, from the corresponding media agent, the information identifying the primary storage location and the corresponding destination storage location.

11. The method of claim 9, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and the corresponding destination storage location; and

allocating resources to the job according to the storage policy.

12. The method of claim 11, wherein the storage policy further specifies a de-duplication rule for the job.

13. The method of claim 9, further comprising:

maintaining, for the primary storage location, information identifying the first subset of data and a first group of destination storage locations at which units making up the first subset of data are stored, and the second subset of data and a second group of destination storage locations at which units making up the second subset of data are stored.

14. The method of claim 9, wherein the at least two different streams comprise a first data streams configured to transfer data over a first host bus adapter and a second data streams configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.

15. The method of claim 9, wherein the primary storage location comprises a source file name.

16. A method of restoring data from a secondary storage to a primary storage, the data having been copied from the primary storage to the secondary storage in multiple subsets based on application-level characteristics tagged to data of each subset, the method comprising:

allocating a job for transferring the data from the secondary storage to the primary storage, wherein the secondary storage comprises a plurality of storage locations, and wherein each storage location stores at least one of the multiple subsets of the data;

determining, for each unit of data to be transferred from the secondary storage to the primary storage, information identifying a primary storage location for the unit of data to which the unit of data is to be restored;

dividing the job into multiple streams based on the application-level characteristics tagged to the data of each subset,

wherein the multiple streams include at least one stream associated with data tagged with an application-level characteristic of user data and at least one stream associated with data tagged with an application-level characteristic of control data, index data, or log data,

wherein each stream has a corresponding data path for carrying data, and

wherein each data path is associated with a corresponding and different media agent and a corresponding and different storage location from the plurality of storage locations of the secondary storage;

assigning the data path for restoring one of the subset of the data stored at the corresponding storage location; and

restoring the multiple subsets of data,

wherein at least two subsets of data are restored using two different data paths.

17. The method of claim 16, further including:

providing, to the corresponding media agent, the information identifying the primary storage location and the corresponding destination storage location.

18. The method of claim 16, further including:

receiving, for the job, a storage policy that specifies a rule for assigning a media agent and the corresponding destination storage location; and

allocating resources to the job according to the storage policy.

19. The method of claim 18, wherein the storage policy further specifies a de-duplication rule for the job, wherein the rule specifies at least one of a directory to be used for de-duplication and a hash function to be used for de-duplication.

20. The method of claim 16, wherein the at least two different streams comprise a first data streams configured to transfer data over a first host bus adapter and a second data streams configured to transfer data over a second host bus adapter that uses a different bus technology than the first host bus adapter.