Systems and methods of data storage management, such as dynamic data stream allocation

• US 8,468,538 B2
• Filed: 03/31/2008
• Issued: 06/18/2013
• Est. Priority Date: 12/22/2006
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Attarde, Deepak R., Gokhale, Parag, Klose, Michael F.
• Primary Examiner(s): Puente, Emerson
• Assistant Examiner(s): Lee, Adam
• Application Number:
  US12/060,151
• Publication Number:
  US 20080288948A1
• Time in Patent Office: 1,905 Days
• Field of Search: None
• US Class Current: 718/105
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A system and method for choosing a stream to transfer data is described. In some cases, the system reviews running data storage operations and chooses a data stream based on the review. In some cases, the system chooses a stream based on the load of data to be transferred.

Claims

1. A computer-implemented method of adapting data storage operations to meet a predetermined time window, the method comprising:

identifying the predetermined time window allocated for a backup operation, wherein the backup operation comprises two or more backup jobs, and wherein each of the two or more backup jobs include at least one data copy operation;

assigning at least one backup job in the backup operation to a first data stream, wherein the first data stream comprises a first media agent and a first storage device;

assigning at least one backup job in the backup operation to a second data stream, the second data stream comprising a second media agent and a second storage device,

wherein the first media agent is different from the second media agent and the first storage device is different from the second storage device;

during the predetermined time window, monitoring the performance of the first and second data streams;

during the predetermined time window, predicting at predetermined time intervals, based at least on the monitored performance of the first and second data streams, whether all of the two or more backup jobs in the backup operation will complete during the predetermined time window; and

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window,

re-assigning at least one backup job from the first data stream to the second data stream,

ordering the two or more backup jobs based on a priority determined from types of data within the two or more backup jobs;

performing backup jobs with higher priority before performing backup jobs having lower priority, and

changing speed of a data backup job based on the determined priority.

2. The method of claim 1, further comprising:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, allocating additional storage resources to the backup operation.

3. The method of claim 1, further comprising:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, selecting a subset of backup jobs and completing the selected backup jobs before performing the unselected backup jobs.

4. The method of claim 1 wherein predicting whether all of the two or more backup jobs in the backup operation will complete during the predetermined time window is performed continuously during the predetermined time window.

5. The method of claim 1, further comprising:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, transferring some of the two or more backup jobs to a future backup operation that is performed during a future time window different than the predetermined time window.

6. The method of claim 1, further comprising:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, ordering the two or more backup jobs based on a priority determined from user input; and

performing backup jobs with higher priority before performing backup jobs having lower priority.

7. A non-transitory computer-readable medium storing instruction that, when executed by at least one data processing device, performs a method of adapting data storage operations to meet a predetermined time window comprising:

identifying the predetermined time window allocated for a backup operation,

wherein the backup operation comprises two or more backup jobs;

assigning at least one backup job in the backup operation to a first data stream, the first data stream comprising a first media agent and a first storage device;

assigning at least one backup job in the backup operation to a second data stream, the second data stream comprising a second media agent and a second storage device,

wherein the first media agent is different from the second media agent and the first storage device is different from the second storage device;

at one or more points during the predetermined time window, monitoring the performance of the first and second data streams;

during the predetermined time window, predicting at predetermined time intervals, based at least on the monitored performance of the first and second data streams, whether all of the two or more backup jobs in the backup operation will complete during the predetermined time window; and

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window,

re-assigning at least one backup job from the first data stream to the second data stream,

ordering the two or more backup jobs based on a priority determined from types of data within the two or more backup jobs;

performing backup jobs with higher priority before performing backup jobs having lower priority, and

changing speed of a data backup job based on the determined priority.

8. The computer-readable medium of claim 7, wherein the method further comprises:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, allocating additional storage resources to the backup operation.

9. The computer-readable medium of claim 7, wherein the method further comprises:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, selecting a subset of backup jobs and completing the selected backup jobs before performing the unselected backup jobs.

10. The computer-readable medium of claim 7 wherein predicting whether all of the two or more backup jobs in the backup operation will complete during the predetermined time window is performed continuously during the predetermined time window.

11. The computer-readable medium of claim 7, wherein the method further comprises:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, transferring some of the two or more backup jobs to a future backup operation that is performed during a future time window different than the predetermined time window.

12. The computer-readable medium of claim 7, wherein the method further comprises:

when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window, ordering the two or more backup jobs based on a priority determined from user input; and

performing backup jobs with higher priority before performing backup jobs having lower priority.

13. A system for adapting data storage operations to meet a backup predetermined time window, the method comprising:

means for identifying the predetermined time window allocated for a backup operation, wherein the backup operation comprises two or more backup jobs;

means for assigning at least one backup job in the backup operation to a first data stream, the first data stream comprising a first media agent and a first storage device;

means for assigning at least one backup job in the backup operation to a second data stream, the second data stream comprising a second media agent and a second storage device,

wherein the first media agent is different from the second media agent and the first storage device is different from the second storage device;

means for monitoring the performance of the first and second data streams at one or more points during the predetermined time window;

means for predicting at predetermined time intervals, during the predetermined time window, based at least on the monitored performance of the first and second data streams, whether all of the two or more backup jobs in the backup operation will complete during the predetermined time window; and

means for re-assigning at least one backup job from the first data stream to the second data stream when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window,

means for ordering the two or more backup jobs based on a priority determined from types of data within the two or more backup jobs;

means for performing backup jobs with higher priority before performing backup jobs having lower priority, and

means for changing speed of a data backup job based on the determined priority.

14. The system of claim 13, further comprising:

means for allocating additional storage resources to the backup operation when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window.

15. The system of claim 13, further comprising:

means for selecting a subset of backup jobs and completing the selected backup jobs before performing the unselected backup jobs when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window.

16. The system of claim 13, further comprising:

means for transferring some of the two or more backup jobs to a future backup operation that is performed during a future time window different than the predetermined time window, wherein the transferring occurs when it is predicted that not all of the two or more backup jobs in the backup operation will complete during the predetermined time window.