Dynamic triggering of block-level backups based on block change thresholds and corresponding file identities using communication pathways between co-resident data agents

• US 10,489,249 B2
• Filed: 09/20/2016
• Issued: 11/26/2019
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Bangalore, Prashanth Nagabhushana, Vullupala, Shankar Reddy, Bhavanam, Sravanthi, Pamarthi, Vinay K., Singireesu, Balaji Teja
• Primary Examiner(s): Vital, Pierre M
• Assistant Examiner(s): Antoine, Alicia M
• Application Number:
  US15/270,909
• Publication Number:
  US 20170091045A1
• Time in Patent Office: 1,162 Days
• Field of Search: 707625
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/156 Query results presentation
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

A data storage management approach is disclosed that performs backup operations flexibly, based on a dynamic scheme of monitoring block changes occurring in production data. The illustrative system monitors block changes based on certain block-change thresholds and triggers block-level backups of the changed blocks when a threshold is passed. Block changes may be monitored in reference to particular files based on a reverse lookup mechanism. The illustrative system also collects and stores historical information on block changes, which may be used for reporting and predictive analysis.

Claims

1. A method for triggering block-level backups based on change thresholds to individual data files, the method comprising:

obtaining, by a first data agent that tracks an application, file-to-data-block relationship information about data files that are accessible to the tracked application in a file system,

wherein the application, the file system, and the first data agent execute on a first computing device comprising one or more processors and non-transitory computer memory,

wherein the first data agent obtains the file-to-data-block relationship information from a second data agent that tracks the file system and also executes on the first computing device, and

wherein the file-to-data-block relationship information identifies a first set of data blocks that store contents of the data files on a first data storage device;

monitoring, by the first data agent, write operations performed by the tracked application to the first data storage device;

maintaining, by the first data agent, a representation of changed data blocks resulting from the write operations performed by the tracked application;

periodically transmitting, by the first data agent, a time-stamped version of the representation of changed data blocks to an index service that executes on a second computing device comprising one or more processors and non-transitory computer memory;

by the index service, keeping count of changed data blocks over time based on a plurality of time-stamped versions of the representation of changed data blocks received from the first data agent;

by the index service, indexing the file-to-data-block relationship information into a data-block-to-file mapping; and

when a threshold measure of changed data blocks is passed by a second set of data blocks changed by the tracked application among the first set of data blocks, wherein the second set of data blocks store contents of a first data file among the data files accessible to the tracked application but do not store contents of other of the data files accessible to the tracked application,

performing at least in part by the first data agent a block-level backup of the second set of data blocks,

wherein the block-level backup is based on changes to the first data file and not to other of the data files accessible to the tracked application and excludes unchanged data blocks that are monitored by the first data agent, and

wherein the index service using the data-block-to-file mapping and the keeping count of the changed data blocks over time determines that the threshold measure of changed data blocks is passed by the first data file.

2. The method of claim 1 wherein the threshold measures changed data blocks, over a pre-determined period of time, to the first data file among the data files accessible to the tracked application.

3. The method of claim 1 wherein the threshold measures changed data blocks by the tracked application over a pre-determined period of time,

thereby resulting in the block-level backup of the second set of data blocks further being based on changes by the tracked application in the pre-determined period of time.

4. The method of claim 1 wherein the threshold measures changed data blocks in a storage volume in the first data storage device;

wherein the second set of data blocks are stored in the storage volume and not elsewhere in the first data storage device; and

thereby resulting in the block-level backup of the second set of data blocks further being based on changes by the tracked application to the storage volume.

5. The method of claim 1 wherein the threshold measures a rate of change of data blocks changed by the tracked application,

thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change of data blocks changed by the tracked application.

6. The method of claim 1 wherein the threshold measures a rate of change of data blocks to the first data file among the data files accessible to the tracked application,

and

thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change to the first data file.

7. The method of claim 1 wherein the threshold measures changed data blocks in the file system,

thereby resulting in the block-level backup of the second set of data blocks further being based on changes to the file system.

8. The method of claim 1 wherein the threshold measures a rate of change of data blocks changed in the file system,

thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change to the file system.

9. The method of claim 1 wherein the threshold measures a rate of change of data blocks changed in a storage volume in the first data storage device;

wherein the second set of data blocks are stored in the storage volume and not elsewhere in the first data storage device; and

thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change to the storage volume.

10. The method of claim 1 wherein the first data agent is a component of a data storage management system that is managed by a storage manager;

and

wherein the first data agent participates in the block-level backup in response to one or more instruction received from the storage manager after the index service executing on the second computing device determines that the threshold has been passed.

11. The method of claim 1 wherein the first data agent is a component of a data storage management system that is managed by a storage manager;

and

further comprising;

receiving, by the index service from the first data agent, the plurality of time-stamped versions of the representation of changed data blocks maintained by the first data agent; and

in response to a query received from the storage manager,

determining, by the index service executing on the second computing device,

based at least in part on (a) the file-to-data-block relationship information and (b) the plurality of time-stamped versions of the representation of changed data blocks resulting from the write operations performed by the tracked application,

which one or more data files in the file system comprise the second set of data blocks, thereby performing a reverse lookup of the block-level changes tracked by the first data agent.

12. A non-transitory computer-readable medium storing instructions that, when executed by a first computing device comprising one or more processors and non-transitory computer memory, cause the first computing device to perform operations comprising:

obtaining, by a first data agent that executes on the first computing device and which tracks an application, file-to-data-block relationship information about data files that are accessible to the tracked application in a file system,

wherein the application, the file system, and the first data agent execute on the first computing device,

wherein the first data agent obtains the file-to-data-block relationship information from a second data agent that tracks the file system and also executes on the first computing device, and

wherein the file-to-data-block relationship information identifies a first set of data blocks that store contents of the data files on a first data storage device;

monitoring, by the first data agent, write operations performed by the tracked application to the first data storage device;

maintaining, by the first data agent, a representation of changed data blocks resulting from the write operations performed by the tracked application;

periodically transmitting, by the first data agent, a time-stamped version of the representation of changed data blocks to an index service that executes on a second computing device comprising one or more processors and non-transitory computer memory,

wherein the index service keeps count of changed data blocks over time based on a plurality of time-stamped versions of the representation of changed data blocks received from the first data agent, and

further wherein the index service indexes the file-to-block relationship information into a data-block-to-file mapping; and

when a threshold measure of changed data blocks is passed by a second set of data blocks changed by the tracked application among the first set of data blocks, wherein the second set of data blocks store contents of a first data file among the data files accessible to the tracked application and do not store contents of other of the data files,

performing at least in part by the first data agent a block-level backup of the second set of data blocks,

wherein the block-level backup is based on changes to the first data file and not to other of the data files accessible to the tracked application and excludes unchanged data blocks that are monitored by the first data agent, and

wherein a determination that the threshold measure is passed for the first data file is made by the index service (a) using the data-block-to-file mapping and (b) keeping count of the changed data blocks over time based on the plurality of time-stamped versions of the representation of changed data blocks received from the first data agent.

13. The non-transitory computer-readable medium of claim 12, wherein the threshold measures changed data blocks, over a pre-determined period of time, to the first data file among the data files accessible to the tracked application.

14. The non-transitory computer-readable medium of claim 12, wherein the threshold measures changed data blocks by the tracked application over a pre-determined period of time, thereby resulting in the block-level backup of the second set of data blocks further being based on changes by the tracked application in the pre-determined period of time.

15. The non-transitory computer-readable medium of claim 12, wherein the threshold measures changed data blocks in a storage volume in the first data storage device;

wherein the second set of data blocks are stored in the storage volume and not elsewhere in the first data storage device; and

thereby resulting in the block-level backup of the second set of data blocks further being based on changes by the tracked application to the storage volume.

16. The non-transitory computer-readable medium of claim 12, wherein the threshold measures a rate of change of data blocks changed by the tracked application, thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change of data blocks changed by the tracked application.

17. The non-transitory computer-readable medium of claim 12, wherein the threshold measures a rate of change of data blocks to the first data file among the data files accessible to the tracked application, thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change to the first data file.

18. The non-transitory computer-readable medium of claim 12, wherein the threshold measures changed data blocks in the file system, thereby resulting in the block-level backup of the second set of data blocks further being based on changes to the file system.

19. The non-transitory computer-readable medium of claim 12, wherein the threshold measures a rate of change of data blocks changed in the file system, thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change to the file system.

20. The non-transitory computer-readable medium of claim 12, wherein the threshold measures a rate of change of data blocks changed in a storage volume in the first data storage device;

wherein the second set of data blocks are stored in the storage volume and not elsewhere in the first data storage device; and

thereby resulting in the block-level backup of the second set of data blocks further being based on the rate of change to the storage volume.