Synchronizing selected portions of data in a storage management system

• US 9,928,005 B2
• Filed: 01/23/2017
• Issued: 03/27/2018
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Sinha, Prosenjit
• Primary Examiner(s): Faal, Baboucarr
• Application Number:
  US15/412,645
• Publication Number:
  US 20170131912A1
• Time in Patent Office: 428 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Disclosed methods and systems leverage resources in a storage management system to partially synchronize primary data files based on synchronizing selected portions thereof without regard to changes that may be occurring in other non-synchronized portions. Accordingly, a number of primary data files may be partially synchronized by synchronizing designated portions thereof via auto-restore operations from backup data. This approach relies on storage management resources to designate portions of source data that is to be kept synchronized across any number of targets; detect changes to the designated portions; back up changes to secondary storage; and distribute the changes from secondary storage to the associated targets, with minimal impact to the primary data environment. The approach may be mutually applied, so that changes in any one of an associated group of source data files may be likewise detected, backed up, and distributed to the other members of the group.

Claims

1. A method for partially synchronizing primary data files based on backup and restore operations of portions thereof in a storage management system, the method comprising:

storing, by a storage manager that manages storage operations in the storage management system, first metadata that identifies a first portion of a first primary data file, second metadata that identifies a second portion of a second primary data file, and an association between the first portion and the second portion, wherein the first primary data file and the second primary data files are not copies of each other;

based on the association between the first portion and the second portion, maintaining, by the storage manager, partial synchronization between the first primary data file and the second primary data file relative to the first portion and the second portion, respectively, wherein the maintaining of the partial synchronization comprises;

receiving, by the storage manager, a first indication of a change to the first portion of the first primary data file,

instructing a first data agent and a media agent executing in the storage management system to back up the first portion of the first primary data file to a secondary copy of the first portion, wherein the secondary copy of the first portion is stored in a first backup format to a storage device associated with the media agent,

instructing the media agent and a second data agent that is associated with the second primary data file to restore the secondary copy of the first portion, from the storage device, to the second portion of the second primary data file,

thereby transforming the secondary copy of the first portion from the first backup format to a format accessible as primary data to an application associated with the second data agent,

receiving, by the storage manager, a second indication of a change to the second portion of the second primary data file,

instructing the second data agent and the media agent to back up the second portion of the second primary data file to a secondary copy of the second portion, wherein the secondary copy of the second portion is stored in a second backup format to the storage device associated with the media agent, and

instructing the media agent and a first data agent that is associated with the first primary data file to restore the secondary copy of the second portion to the first portion of the first primary data file,

thereby transforming the secondary copy of the second portion from the second backup format to a format accessible as primary data to an application associated with the first data agent; and

wherein after the backup and restore operations are complete, the first primary data file and the second primary data file are partially synchronized relative to the first portion of the first primary data file and the second portion of the second primary data file, respectively, regardless of changes to other portions of the first primary data file that fall outside the first portion, and also regardless of any changes to other portions of the second primary data file that fall outside the second portion.

2. The method of claim 1 wherein the first backup format is the same as the second backup format.

3. The method of claim 1 wherein the first backup format is different from the second backup format.

4. The method of claim 1 wherein the first data agent detects the change to the first portion of the first primary data file based on an operating-system change-log in the first computing device.

5. The method of claim 1 wherein the maintaining is based on backup and restore operations in the storage management system, wherein a given backup operation is initiated by the storage manager.

6. The method of claim 1 wherein the instructing of a given data agent to backup a corresponding portion of a respective primary data file is based on an indication of change in the corresponding portion since a preceding restore operation thereto.

7. The method of claim 1 wherein the instructing of a given data agent to backup a corresponding portion of a respective primary data file is based on an indication of change in the corresponding portion since a preceding backup therefrom.

8. The method of claim 1 wherein the maintaining of the partial synchronization further comprises transmitting, by the storage manager to the first data agent an identifier of the first portion of the first primary data file.

9. A computer-readable medium, excluding transitory propagating signals, storing instructions that, when executed by at least one computing device comprising one or more processors and computer-readable memory, cause the computing device to perform a method for partially synchronizing primary data files based on backup and restore operations of portions thereof, the method comprising:

storing, by the computing device, an association between a first portion of a first primary data file and a second portion of a second primary data file, wherein the first primary data file and the second primary data files are not copies of each other;

based on the association between the first portion and the second portion, maintaining, by the computing device, partial synchronization between the first primary data file and the second primary data file relative to the first portion and the second portion, respectively, wherein the maintaining of the partial synchronization comprises;

based on a first indication that the first portion of the first primary data file has changed, instructing a first data agent and a media agent to back up the first portion of the first primary data file to a secondary copy of the first portion, wherein the secondary copy of the first portion is stored in a first backup format to a storage device associated with the media agent,

instructing the media agent and a second data agent that is associated with the second primary data file to restore the secondary copy of the first portion from the storage device to the second portion of the second primary data file,

thereby transforming the secondary copy of the first portion from the first backup format to a format accessible as primary data to an application associated with the second data agent,

based on a second indication that the second portion of the second primary data file has changed, instructing the second data agent and the media agent to back up the second portion of the second primary data file to a secondary copy of the second portion, wherein the secondary copy of the second portion is stored in a second backup format to the storage device associated with the media agent, and

instructing the media agent and the first data agent to restore the secondary copy of the second portion to the first portion of the first primary data file,

thereby transforming the secondary copy of the second portion from the second backup format to a format accessible as primary data to an application associated with the first data agent; and

wherein after the backup and restore operations are complete, the first primary data file and the second primary data file are partially synchronized relative to the first portion of the first primary data file and the second portion of the second primary data file, respectively, regardless of changes to other portions of the first primary data file that fall outside the first portion, and also regardless of changes to other portions of the second primary data file that fall outside the second portion.

10. The computer-readable medium of claim 9, wherein the first indication is received by the computing device from the first data agent, and wherein the second indication is received by the computing device from the second data agent.

11. The computer-readable medium of claim 9, wherein the maintaining of the partial synchronization further comprises:

instructing the first data agent to monitor the first portion of the first primary data file for changes, and

instructing the second data agent to monitor the second portion of the second primary data file for changes.

12. The computer-readable medium of claim 9, wherein the maintaining of the partial synchronization further comprises:

instructing the first data agent to monitor the first portion of the first primary data file for changes and to notify the computing device thereof, and

instructing the second data agent to monitor the second portion of the second primary data file for changes and to notify the computing device thereof.

13. The computer-readable medium of claim 9, wherein a given secondary copy of a corresponding portion of a respective primary data file is not accessible as primary data to (a) the first application associated with the first data agent, and (b) the second application associated with the second data agent.