Dynamic triggering of block-level backups based on block change thresholds and corresponding file identities in a data storage management system

• US 10,514,986 B2
• Filed: 09/20/2016
• Issued: 12/24/2019
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Bangalore, Prashanth Nagabhushana, Vullupala, Shankar Reddy, Bhavanam, Sravanthi, Pamarthi, Vinay K., Singireesu, Balaji Teja
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US15/271,129
• Publication Number:
  US 20170091047A1
• Time in Patent Office: 1,190 Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/156 Query results presentation
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

A data storage management approach is disclosed that performs backup operations flexibly, based on a dynamic scheme of monitoring block changes occurring in production data. The illustrative system monitors block changes based on certain block-change thresholds and triggers block-level backups of the changed blocks when a threshold is passed. Block changes may be monitored in reference to particular files based on a reverse lookup mechanism. The illustrative system also collects and stores historical information on block changes, which may be used for reporting and predictive analysis.

Claims

1. A system for triggering block-level backups based on change thresholds to individual data files, the system comprising:

a storage manager comprising a computing device that includes one or more processors and non-transitory computer memory;

an index server in communication with the storage manager, wherein the index server includes one or more processors and non-transitory computer memory;

a client computing device in communication with the storage manager and the index server, wherein the client computing device includes one or more processors and non-transitory computer memory for executing an application, a first data agent that tracks the application, a file system, and a second data agent that tracks the file system;

a first data storage device in communication with the client computing device, wherein the first data storage device stores data files accessible by the application,

wherein a first set of data blocks store contents of the data files;

wherein the first data agent is programmed to;

monitor write operations performed by the application to the first data storage device,

identify data blocks changing as a result of the write operations,

maintain a representation of changed data blocks in the first data storage device resulting from the write operations, and

periodically (i) transmit a time-stamped version of the representation of changed data blocks, designated a point-in-time map, to the index server and (ii) re-initialize the representation of changed data blocks maintained by the first data agent;

wherein the second data agent that monitors the file system is programmed to;

transmit to the first data agent file-to-data-block relationship information that identifies the first set of data blocks that store the contents of the data files in the file system that are accessible to the application;

wherein the index server is programmed to;

receive a plurality of point-in-time maps,

keep count of changed data blocks over time based on the plurality of point-in-time maps,

index the file-to-data-block relationship information received from the second data agent into a data-block-to-file mapping,

based on using the data-block-to file mapping, determine that a threshold measure of data block changes in a first of the data files has been passed by a second set of changed data blocks represented in the point-in-time maps, wherein the second set of changed data blocks is a subset of the first set of data blocks, and

notify the storage manager of the threshold having been passed; and

wherein the storage manager is programmed to;

based on receiving notice from the index server that the threshold has been passed, initiate a block-level backup of the second set of changed data blocks,

wherein the block-level backup of the second set of changed data blocks does not back up the first of the data files in its entirety, and

wherein the block-level backup of the second set of changed data blocks is triggered by changes in the first of the data files and not by changes in others of the data files.

2. The system of claim 1 wherein a first storage volume on the first data storage device stores the first set of data blocks that store the contents of the data files in the file system that are accessible to the application;

and wherein the block-level backup of the second set of changed data blocks does not back up the first storage volume in its entirety.

3. The system of claim 1 further comprising:

a report server in communication with the storage manager and the index server, wherein the report server includes one or more processors and non-transitory computer memory; and

wherein the report server is programmed to;

receive a first query about data block changes in the system,

access the plurality of point-in-time maps, and

formulate an answer to the first query based on information extracted from the plurality of point-in-time maps in the index server.

4. The system of claim 3 wherein the first query identifies a second of the data files wherein a third set of data blocks stores contents thereof;

and wherein based at least in part on one or more point-in-time maps and the data-block-to-file mapping in the index server, the report server is further programmed to determine changes that occurred within the second of the data files.

5. The system of claim 3 wherein the first query identifies a third set of data blocks in the system;

and wherein based at least in part on one or more point-in-time maps and the data-block-to-file mapping in the index server, the report server is further programmed to determine which among the data files experienced changes that occurred within the third set of data blocks.

6. A system for data storage management, the system comprising:

a storage manager comprising a computing device that includes one or more processors and non-transitory computer memory;

an index server in communication with the storage manager, wherein the index server includes one or more processors and non-transitory computer memory;

a client computing device in communication with the storage manager and the index server, wherein the client computing device includes one or more processors and non-transitory computer memory for executing an application, a first data agent that tracks the application, a file system, and a second data agent that tracks the file system;

a first data storage device in communication with the client computing device, wherein the first data storage device stores data files accessible by the application, wherein a first set of data blocks store contents of the data files;

wherein the first data agent is programmed to;

monitor write operations performed by the application to the first data storage device,

identify data blocks changing as a result of the write operations,

maintain a representation of changed data blocks in the first data storage device resulting from the write operations,

periodically (i) transmit a time-stamped version of the representation of changed data blocks, designated a point-in-time map, to the index server and (ii) re-initialize the representation of changed data blocks maintained by the first data agent, and

transmit a first query to the second data agent requesting file-to-data-block relationship information about data files in the file system, including at least one data file accessible to the application;

wherein the second data agent is programmed to;

obtain the file-to-data-block relationship information based on a scan of the file system, and

transmit the file-to-data-block relationship information to the first data agent in response to the first query;

wherein the index server is programmed to;

receive a plurality of point-in-time maps,

index the file-to-data-block relationship information into a data-block-to-file mapping,

based on using the data-block-to-file mapping, determine that a threshold measure of changed data blocks in a first of the data files has been passed by a second set of changed data blocks represented in the point-in-time maps, wherein the second set of changed data blocks is a subset of the first set, and

notify the storage manager of the threshold having been passed; and

wherein the storage manager is programmed to;

based on receiving notice from the index server that the threshold has been passed, initiate a block-level backup of the second set of changed data blocks, by triggering the block-level backup based on changes in the first of the data files and not by changes in others of the data files.

7. The system of claim 6 wherein the index server is programmed to receive the threshold measure of changed data blocks from the storage manager.

8. The system of claim 6 wherein the threshold applies to a specific set of data files in the file system including the first of the data files.

9. The system of claim 6 wherein the threshold comprises a rate of change for changed data blocks being monitored.

10. The system of claim 6 wherein the threshold comprises a percentage of changed data blocks being monitored.

11. The system of claim 6 wherein the threshold applies to a pre-defined period of time being monitored.

12. The system of claim 6 wherein the threshold measures a rate of change of data blocks changed by the tracked application;

and thereby resulting in the block-level backup of the second set of data blocks being based on the rate of change of data blocks changed by the tracked application.