Synchronizing selected portions of data in a storage management system

• US 9,952,934 B2
• Filed: 01/20/2015
• Issued: 04/24/2018
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Sinha, Prosenjit
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US14/601,036
• Publication Number:
  US 20160210195A1
• Time in Patent Office: 1,190 Days
• Field of Search: 707625
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Disclosed methods and systems leverage resources in a storage management system to partially synchronize primary data files based on synchronizing selected portions thereof without regard to changes that may be occurring in other non-synchronized portions. Accordingly, a number of primary data files may be partially synchronized by synchronizing designated portions thereof via auto-restore operations from backup data. This approach relies on storage management resources to designate portions of source data that is to be kept synchronized across any number of targets; detect changes to the designated portions; back up changes to secondary storage; and distribute the changes from secondary storage to the associated targets, with minimal impact to the primary data environment. The approach may be mutually applied, so that changes in any one of an associated group of source data files may be likewise detected, backed up, and distributed to the other members of the group.

Claims

1. A method for partially synchronizing primary data files based on using backup and restore operations of associated portions thereof in a storage management system, the method comprising:

receiving, by a first data agent executing on a first client computing device in a storage management system, from a storage manager that manages storage operations in the storage management system, first metadata that identifies a first portion of a first primary data file, wherein the first primary data file comprises data generated by the first client computing device, and wherein the first primary data file is associated with the first data agent;

detecting, by the first data agent, at least one change in the first portion of the first primary data file;

transmitting, by the first data agent to the storage manager, notice of the detected at least one change in the first portion of the first primary data file;

executing, by the first data agent in conjunction with a media agent, as instructed by the storage manager, a backup operation of the first portion of the first primary data file, wherein the backup operation generates a secondary copy of the first portion of the first primary data file, wherein the secondary copy of the first portion is stored by the media agent to a secondary storage device in a backup format that is not directly accessible to the first client computing device;

executing, by a second data agent in conjunction with the media agent, as instructed by the storage manager, a restore operation of the secondary copy to a second portion of a second primary data file,

based on an association for mutual synchronization between the first portion of the first primary data file and the second portion of the second primary data file maintained by the storage manager,

wherein the second primary data file is associated with the second data agent, which executes on a second client computing device in the storage management system,

wherein the restore operation transforms the secondary copy of the first portion from the backup format to a native format that is directly accessible as primary data to the second client computing device; and

wherein the first primary data file and the second primary data file are partially synchronized relative to the associated first portion and the second portion, respectively, based on the backup operation of the first portion and further based on the restore operation of the secondary copy of the first portion to the second portion of the second primary data file.

2. The method of claim 1 wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, regardless of any changes to other portions of the first primary data file that fall outside the first portion, and also regardless of any changes to other portions of the second primary data file that fall outside the second portion.

3. The method of claim 1 wherein the detecting is based at least in part on an operating-system change log residing on the first client computing device.

4. The method of claim 1 wherein the detecting is based at least in part on the first data agent monitoring an operating-system change log, which resides on the first client computing device, for changes to one or more data blocks in the first portion of the first primary data file, wherein the monitoring is based on the received first metadata.

5. The method of claim 1 further comprising:

monitoring, by the first data agent, based on the received first metadata, the first portion of the first primary data file to detect whether one or more changes occur in the first portion.

6. The method of claim 1 further comprising:

monitoring, by the first data agent, based on the received first metadata, an operating-system change log, which resides on the first client computing device, for changes to one or more data blocks in the first portion of the first primary data file.

7. The method of claim 1 wherein the detecting of the at least one change in the first portion of the first primary data file is based at least in part on monitoring of the first portion by the first data agent, wherein the monitoring is based on the received first metadata.

8. The method of claim 1 wherein the detecting of the at least one change in the first portion of the first primary data file is based at least in part on monitoring of the first portion by the first data agent, wherein the monitoring is based at least in part on an operating-system change log residing on the first client computing device.

9. The method of claim 1 further comprising:

monitoring, by the first data agent, based on the received first metadata, the first portion of the first primary data file to detect whether one or more changes occur in the first portion;

monitoring, by the second data agent, based on received second metadata that identifies the second portion of the second primary data file, to detect whether one or more changes occur in the second portion; and

when a change is detected in one of the first portion and the second portion, synchronizing the first portion and the second portion to include the detected change, thereby partially synchronizing the first primary data file and the second data file, respectively;

wherein the synchronizing of the first portion and the second portion comprises (i) backing up the changed one of the first portion and the second portion to a secondary copy, and (ii) restoring the secondary copy to the other one of the first portion and the second portion that was previously unchanged.

10. The method of claim 1 wherein the first portion of the first primary data file and the second portion of the second primary data file are selected by a user of the storage management system.

11. The method of claim 1 further comprising:

detecting, by the second data agent, at least one change in the second portion of the second primary data file;

transmitting, by the second data agent to the storage manager, notice of the detected at least one change in the second portion of the second primary data file;

based on a third instruction received from the storage manager, executing, by the second data agent, at least part of a backup operation of the second portion of the second primary data file, which results in a secondary copy of the second portion of the second primary data file;

based on a fourth instruction received from the storage manager, executing, by the first data agent, at least part of a restore operation of the secondary copy to the first portion of the first primary data file; and

wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, based on the backup operation of the second portion and further based on the restore operation of the secondary copy of the second portion to the first portion of the first primary data file.

12. A method for partially synchronizing primary data files based on backup and restore operations of portions thereof in a storage management system, the method comprising:

receiving, by a first data agent executing on a first client computing device in the storage management system, from a storage manager that manages storage operations in the storage management system, first metadata that identifies a first portion of a first primary data file, wherein the first primary data file comprises data generated by the first client computing device, and wherein the first primary data file is associated with the first data agent;

receiving, by a second data agent executing on a second client computing device in the storage management system, from the storage manager, second metadata that identifies a second portion of a second primary data file, wherein the second primary data file comprises data generated by the second client computing device, and wherein the second primary data file is associated with the second data agent;

monitoring the first portion of the first primary data file, by the first data agent, based on the received first metadata, to detect whether one or more changes occur in the first portion;

monitoring the second portion of the second primary data file, by the second data agent, based on the second received metadata, to detect whether one or more changes occur in the second portion; and

when a change is detected in one of (a) the first portion of the first primary data file and (b) the second portion of the second primary data file, by a respective associated data agent, synchronizing the first portion and the second portion to include the detected change, thereby partially-synchronizing the first primary data file and the second primary data file, respectively, based on an association for mutual synchronization between the first portion of the first primary data file and the second portion of the second primary data file maintained by the storage manager;

wherein the synchronizing of the first portion and the second portion comprises;

(i) backing up by the detecting associated data agent in conjunction with a media agent, as instructed by the storage manager, the changed one of the first portion and the second portion to a secondary copy thereof in a backup format and stored to a secondary storage device associated with the media agent, and

(ii) restoring the secondary copy, from the backup format in the secondary storage device, to the other one of the first portion and the second portion in a respective native format that is accessible as primary data to the respective client computing device,

wherein the restoring is performed, as instructed by the storage manager, by the media agent in conjunction with the data agent executing on the respective client computing device accessing the restored primary data.

13. The method of claim 12 wherein the synchronizing of the first portion and the second portion is initiated by the storage manager.

14. The method of claim 12 further comprising:

when the change is detected in one of the first portion of the first primary data file and the second portion of the second primary data file, by a respective associated data agent, transmitting notice of the detected change by the respective associated data agent to the storage manager; and

wherein the synchronizing of the first portion and the second portion is initiated by the storage manager based on the notice of detected change.

15. A method for partially synchronizing primary data files based on backup and restore operations of portions thereof in a storage management system, the method comprising:

receiving, by a first data agent executing on a first client computing device in the storage management system, from a storage manager that manages storage operations in the storage management system, first metadata that identifies a first portion of a first primary data file, wherein the first primary data file comprises data generated by the first client computing device, and wherein the first primary data file is associated with the first data agent;

detecting, by the first data agent, at least one change in the first portion of the first primary data file;

backing up, by the first data agent in conjunction with a media agent, the first portion of the first primary data file to a secondary copy of the first portion of the first primary data file, wherein the secondary copy is stored in a backup format to a secondary storage device in communication with the media agent;

restoring, by a second data agent in conjunction with the media agent, the secondary copy of the first portion of the first primary data file to a second portion of a second primary data file, based on an association for mutual synchronization between the first portion of the first primary data file and the second portion of the second primary data file maintained by the storage manager, wherein the second primary data file is associated with the second data agent, which executes on a second client computing device in the storage management system,

thereby transforming the secondary copy of the first portion from the backup format to a native format that is accessible as primary data to the second client computing device; and

wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, based on the backing up of the first portion and further based on the restoring of the secondary copy of the first portion to the second portion of the second primary data file.

16. The method of claim 15 wherein the backing up is executed based on instructions received by the first data agent and the media agent from the storage manager.

17. The method of claim 15 wherein the backing up is executed based on instructions received by the first data agent and the media agent from the storage manager based on notice of detected at least one change received by the storage manager from the first data agent.

18. The method of claim 15 wherein the restoring is executed based on instructions received by the second data agent and the media agent from the storage manager, based on completion of the backing up operation.