Data protection scheduling, such as providing a flexible backup window in a data protection system

• US 8,769,048 B2
• Filed: 06/18/2008
• Issued: 07/01/2014
• Est. Priority Date: 06/18/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kottomtharayil, Rajiv
• Primary Examiner(s): Gorney, Boris
• Application Number:
  US12/141,776
• Publication Number:
  US 20090320029A1
• Time in Patent Office: 2,204 Days
• Field of Search: 709/219, 711/167
• US Class Current: 709/219
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 2209/506 Constraint
• CPC Class Codes: G06F 9/505 considering the load

Abstract

A data protection scheduling system provides a flexible or rolling data protection window that analyzes various criteria to determine an optimal or near optimal time for performing data protection or secondary copy operations. While prior systems may have scheduled backups at an exact time (e.g., 2:00 a.m.), the system described herein dynamically determines when to perform the backups and other data protection storage operations, such as based on network load, CPU load, expected duration of the storage operation, rate of change of user activities, frequency of use of affected computer systems, trends, and so on.

Claims

1. A method for dynamically determining a schedule for performing a data storage operation, the method comprising:

receiving a request to perform a storage operation;

determining whether the storage operation is exempt from dynamically determined scheduling, and

when the storage operation is not exempt from dynamically determined scheduling, then;

applying one or more performance criteria to dynamically determine a time to perform the storage operation,

wherein the applying one or more performance criteria includes adding weighting to the storage operation to change a previously determined schedule of operations, if the storage operation did not get processed previously;

scheduling the data storage operation based on the one or more applied performance criteria, and

monitoring usage of resources after starting the storage operation for dynamically altering the schedule for performing the storage operation;

wherein one or more of the applied performance criteria are unassociated with an amount of data to store under the storage operations, and

wherein at least one of the one or more applied performance criteria relate to a predicted usage of computing resources in performing the storage operation;

wherein scheduling the data storage operation comprises;

determining a storage window at least partly based on a future network load;

determining a schedule for completing multiple requested data storage operations within the storage window,

comparing, during execution of the schedule, the storage window with an estimated time remaining to complete data storage operations;

delaying, based on the comparison, storage of some types of data in order to store other types of data within the storage window; and

sending some data storage jobs to other data storage systems;

when the storage operation is exempt from dynamically determined scheduling, then;

performing the storage operation in an order in which hard coded storage operations to be completed within a time window are performed first, followed by

performing storage operations having a deadline of completion less than a threshold deadline.

2. The method of claim 1 wherein the request includes a deadline for performing the storage operation based on past trends of storage operations.

3. The method of claim 1 wherein the storage operation includes one of storing, receiving and preparing data to be stored, copied or backed up, and the request includes a desired frequency for performing the storage operation.

4. The method of claim 1 wherein the request includes a request criterion and wherein applying at least one performance criterion takes into account the request criterion.

5. The method of claim 1 wherein at least one performance criterion of the one or more performance criteria relates to a time when resources of the computer systems affected by the storage operation are unlikely to be in use, a time when the network will have a relatively lower load, or a time when a media library to which data will be transferred by the storage operation is not in use.

6. The method of claim 1 including reviewing the determined schedule before the storage operation starts and changing the schedule based on other scheduled storage operations.

7. The method of claim 1 including reviewing the determined schedule before the storage operation starts and changing the schedule based on information received after the storage operation was last scheduled.

8. The method of claim 1 wherein scheduling the data storage operation avoids at least some conflicts between computer systems over resources needed to perform the storage operation.

9. The method of claim 1, further including:

looking at a period of time to identify a lull in system resources;

making predictions about usage of the system resources; and

starting a new storage operation based on the predictions.

10. The method of claim 1, further including adding weighting to an expiring job and a job that was started but not completed.

11. The method of claim 1 further including:

dynamically determining the schedule based on the resources required by each request and a predicted availability of resources at various times within the storage window.

12. A non-transitory, computer-readable medium storing instructions, which, when executed by at least one computing device, performs a method for dynamically determining a schedule for performing a data storage operation, the method comprising:

receiving a request to perform a storage operation;

determining if the storage operation is exempt from dynamically determined scheduling, and

when the storage operation is not exempt from dynamically determined scheduling, then;

applying one or more performance criteria to dynamically determine a time to perform the storage operation,

wherein the applying one or more performance criteria includes adding weighting to the storage operation to change a previously determined schedule of operations, if the storage operation did not get processed previously;

scheduling the data storage operation based on the one or more applied performance criteria, and

monitoring usage of resources after starting the data storage operation for dynamically altering the schedule for performing the storage operation;

wherein one or more of the applied performance criteria are unassociated with an amount of data to store under the storage operations, and

wherein one or more applied performance criteria relate to a predicted usage of computing resources in performing the storage operation;

wherein scheduling the data storage operation comprises;

determining a storage window at least partly based on a future network load;

determining a schedule that is sufficient to complete multiple requested data storage operations within the storage window,

comparing, during execution of the schedule, the storage window with an estimated time remaining to complete data storage operations;

delaying, based on the comparison, storage of some types of data in order to store other types of data within the storage window; and

sending some data storage jobs to other data storage systems;

when the storage operation is exempt from dynamically determined scheduling, then;

performing the storage operation in an order in which hard coded storage operations to be completed within a time window are performed first, followed by

performing storage operations having a deadline of completion less than a threshold deadline.

13. The non-transitory, computer-readable medium of claim 12 wherein the request includes a deadline for performing the storage operation based on past trends of storage operations.

14. The non-transitory, computer-readable medium of claim 12 wherein the request includes a desired frequency for performing the storage operation.

15. The non-transitory, computer-readable medium of claim 12 wherein the request includes a request criterion and wherein applying at least one performance criterion takes into account the request criterion.

16. The non-transitory, computer-readable medium of claim 12 wherein at least one performance criterion of the one or more performance criteria relates to one of a time when resources of the computer systems affected by the storage operation are unlikely to be in use, a time when the network will have a relatively lower load, and a time when a media library to which data will be transferred by the storage operation is not in use.

17. The non-transitory, computer-readable medium of claim 12 including reviewing the determined schedule before the storage operation starts and changing the schedule based on other scheduled storage operations.

18. The non-transitory, computer-readable medium of claim 12 including reviewing the determined schedule before the storage operation starts and changing the schedule based on information received after the storage operation was last scheduled.

19. The non-transitory, computer-readable medium of claim 12 wherein scheduling the data storage operation avoids at least some conflicts between computer systems over resources needed to perform the storage operation.

20. A system to dynamically determine a schedule for performing a data storage operation, the system comprising:

a processor; and

a memory for storing computer-executable instructions, said instructions, when executed by the processor, causing the processor to implement a method comprising;

receiving a request to perform a storage operation;

determining if the storage operation is exempt from dynamically determined scheduling, and

when the storage operation is not exempt from dynamically determined scheduling, then;

applying one or more performance criteria to dynamically determine a time to perform the storage operation,

wherein the applying one or more performance criteria includes adding weighting to the storage operation to change a previously determined schedule of operations, if the storage operation did not get processed previously;

scheduling the data storage operation based on the one or more applied performance criteria, and

monitoring usage of resources after starting the data storage operation for dynamically altering the schedule for performing the storage operation;

wherein one or more applied performance criteria are unassociated with an amount of data to store under the storage operations, and

wherein one or more applied performance criteria relate to a predicted usage of computing resources in performing the storage operation;

wherein the scheduling the data storage operation comprises;

determining a storage window at least partly based on a future network load;

determining a schedule for completing multiple requested data storage operations within the storage window,

comparing, during execution of the schedule, the storage window with an estimated time remaining to complete data storage operations;

delaying, based on the comparison, storage of some types of data in order to store other types of data within the storage window; and

sending some data storage jobs to other data storage systems;

when the storage operation is exempt from dynamically determined scheduling, then;

performing the storage operation in an order in which hard coded storage operations to be completed within a time window are performed first, followed by

performing storage operations having a deadline of completion less than a threshold deadline.

21. The system of claim 20 wherein the request includes a deadline for performing the storage operation based on past trends of storage operations.

22. The system of claim 20 wherein the request includes a desired frequency for performing the storage operation.

23. The system of claim 20 wherein the request includes a request criterion and wherein the means for applying at least one performance criterion takes into account the request criterion.

24. The system of claim 20 wherein at least one performance criterion of the one or more performance criteria relates to a time when resources of the computer systems affected by the storage operation are unlikely to be in use or a time when the network will have a relatively lower load, or a time when a media library to which data will be transferred by the storage operation is not in use.

25. The system of claim 20, wherein the method further comprises reviewing the determined schedule before the storage operation starts and changing the schedule based on other scheduled storage operations.

26. The system of claim 20, wherein the method further comprises reviewing the determined schedule before the storage operation starts and changing the schedule based on information received after the storage operation was last scheduled.

27. The system of in claim 20 wherein the scheduling the data storage operation avoids at least some conflicts between computer systems over resources needed to perform the storage operation.

28. The system of claim 20 wherein the processor controls a database agent to store historical data about an availability of resources for performing storage operations.

29. The system of claim 20, wherein the processor implements adding weighting to an expiring job and a job that was started but not completed.

30. The system of claim 20, wherein the processor further:

looks at a period of time to identify a lull in system resources;

makes predictions about usage of the system resources; and

starts a new storage operation based on the predictions.