Synchronizing selected portions of data in a storage management system

• US 10,713,122 B2
• Filed: 03/05/2018
• Issued: 07/14/2020
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Sinha, Prosenjit
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US15/912,349
• Publication Number:
  US 20180260282A1
• Time in Patent Office: 862 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Disclosed methods and systems leverage resources in a storage management system to partially synchronize primary data files based on synchronizing selected portions thereof without regard to changes that may be occurring in other non-synchronized portions. Accordingly, a number of primary data files may be partially synchronized by synchronizing designated portions thereof via auto-restore operations from backup data. This approach relies on storage management resources to designate portions of source data that is to be kept synchronized across any number of targets; detect changes to the designated portions; back up changes to secondary storage; and distribute the changes from secondary storage to the associated targets, with minimal impact to the primary data environment. The approach may be mutually applied, so that changes in any one of an associated group of source data files may be likewise detected, backed up, and distributed to the other members of the group.

Claims

1. A method for partially synchronizing primary data files based on backing up and restoring associated portions thereof in a storage management system, the method comprising:

receiving, by a first data agent executing on a first computing device, from a storage manager that manages storage operations in the storage management system, identifying information for a first portion of a first primary data file, wherein the first primary data file is associated with the first data agent and comprises data generated by the first computing device;

detecting, by the first data agent, at least one change in the first portion of the first primary data file;

notifying the storage manager of the detected at least one change in the first portion of the first primary data file;

backing up, by the first data agent in conjunction with a media agent, the first portion of the first primary data file resulting in a secondary copy of the first portion of the first primary data file, wherein the secondary copy of the first portion is stored by the media agent in a backup format that is not directly accessible to the first computing device;

restoring, by a second data agent in conjunction with the media agent, the secondary copy of the first portion to a second portion of a second primary data file, thereby transforming the secondary copy of the first portion from the backup format to a native format that is directly accessible as primary data to a second computing device that executes the second data agent;

wherein the storage manager maintains an association for mutual synchronization between the first portion of the first primary data file and the second portion of the second primary data file; and

wherein after the backing up and the restoring operations the first primary data file and the second primary data file are partially synchronized relative to the associated first portion and the second portion, respectively.

2. The method of claim 1, wherein the storage manager initiates the backing up based on the notifying of the detected at least one change in the first portion of the first primary data file by the first data agent.

3. The method of claim 1, wherein the media agent stores the secondary copy to a storage device in the storage management system.

4. The method of claim 1, wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, regardless of any changes to other portions of the first primary data file that fall outside the first portion, and regardless of any changes to other portions of the second primary data file that fall outside the second portion.

5. The method of claim 1, wherein the detecting is based at least in part on an operating-system change log residing on the first computing device.

6. The method of claim 1, wherein the detecting is based at least in part on the first data agent monitoring an operating-system change log on the first computing device for changes to one or more data blocks in the first portion of the first primary data file.

7. The method of claim 1 further comprising:

monitoring, by the first data agent, based on the received identifying information, the first portion of the first primary data file to detect whether one or more changes occur in the first portion.

8. The method of claim 1, wherein the detecting of the at least one change in the first portion of the first primary data file is based on monitoring of the first portion by the first data agent based on the received identifying information.

9. The method of claim 1 further comprising:

monitoring, by the first data agent, based on the received identifying information, the first portion of the first primary data file to detect whether one or more changes occur in the first portion;

monitoring, by the second data agent, based on other identifying information that identifies the second portion of the second primary data file, to detect whether one or more changes occur in the second portion; and

when one or more changes are detected in one of;

the first portion and the second portion, synchronizing the first portion and the second portion to include the detected one or more changes, resulting in the first primary data file and the second primary data file being partially synchronized relative to the first portion and the second portion, respectively,

wherein the synchronizing of the first portion and the second portion comprises;

(i) backing up the changed one of;

the first portion and the second portion to an other secondary copy, and (ii) restoring the other secondary copy to the other one of;

the first portion and the second portion that was previously unchanged.

10. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, by a first data agent executing on first computing device, from a storage manager that manages storage operations in a storage management system, identifying information for a first portion of a first primary data file, wherein the first primary data file is associated with the first data agent and comprises data generated by the first computing device;

detecting, by the first data agent, at least one change in the first portion of the first primary data file;

notifying the storage manager of the detected at least one change in the first portion of the first primary data file;

backing up, by the first data agent in conjunction with a media agent, the first portion of the first primary data file resulting in a secondary copy of the first portion of the first primary data file, wherein the secondary copy of the first portion is stored by the media agent in a backup format that is not directly accessible to the first computing device;

causing a second data agent in conjunction with the media agent, to restore the secondary copy of the first portion to a second portion of a second primary data file, thereby transforming the secondary copy of the first portion from the backup format to a native format that is directly accessible as primary data to a second computing device that executes the second data agent;

wherein an association for mutual synchronization between the first portion of the first primary data file and the second portion of the second primary data file is maintained by the storage manager; and

wherein after the backing up and the restoring operations the first primary data file and the second primary data file are partially synchronized relative to the associated first portion and the second portion, respectively.

11. The non-transitory computer-readable medium of claim 10, wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, regardless of any changes to other portions of the first primary data file that fall outside the first portion, and regardless of any changes to other portions of the second primary data file that fall outside the second portion.

12. The non-transitory computer-readable medium of claim 10, wherein the detecting is based at least in part on an operating-system change log residing on the first computing device.

13. The non-transitory computer-readable medium of claim 10, wherein the detecting is based at least in part on the first data agent monitoring an operating-system change log on the first computing device for changes to one or more data blocks in the first portion of the first primary data file.

14. The non-transitory computer-readable medium of claim 10, wherein the detecting of the at least one change in the first portion of the first primary data file is based on monitoring of the first portion by the first data agent based on the received identifying information.

15. A method comprising:

receiving, by a first data agent executing on a first computing device, from a storage manager that manages storage operations in a storage management system, first information that identifies a first portion of a first primary data file, wherein the first primary data file comprises data generated by the first computing device, and wherein the first primary data file is associated with the first data agent;

receiving, by a second data agent executing on a second computing device, from the storage manager, second information that identifies a second portion of a second primary data file, wherein the second primary data file comprises data generated by the second computing device, and wherein the second primary data file is associated with the second data agent;

based on the first information, monitoring the first portion of the first primary data file, by the first data agent, to detect whether one or more changes occur in the first portion;

based on the second information, monitoring the second portion of the second primary data file, by the second data agent, to detect whether one or more changes occur in the second portion;

when one or more first changes are detected by the first data agent in the first portion of the primary data file, (i) backing up the first portion to a first secondary copy thereof in a backup format that is not accessible as primary data to the first computing device, and (ii) restoring the first secondary copy to the second portion of the second primary data file in a native format that is accessible as primary data to the second computing device;

when one or more second changes are detected by the second data agent in the second portion of the second primary data file, (i) backing up the second portion to a second secondary copy thereof in a backup format that is not accessible as primary data to the second computing device, and (ii) restoring the second secondary copy to the first portion of the first primary data file in a native format that is accessible as primary data to the first computing device; and

wherein the storage manager manages the backing up and restoring operations based on an association for mutual synchronization between the first portion of the first primary data file and the second portion of the second primary data file which is maintained by the storage manager.

16. The method of claim 15, wherein the first primary data file and the second primary data file are partially synchronized by the backing up and restoring operations.

17. The method of claim 15, wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, based on the backing up and the restoring operations.

18. The method of claim 15, wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, based on the backing up and the restoring operations, regardless of whether other changes occur in the first primary data file or the second primary data file outside of the first portion and the second portion respectively.

19. The method of claim 15 further comprising:

when the one or more first changes are detected by the first data agent in the first portion of the first primary data file, notifying the storage manager by the first data agent before the backing up of the first portion is initiated.

20. The method of claim 15, wherein the monitoring of the first portion of the first primary data file by the first data agent is based on an operating-system change log on the first computing device.