Synchronizing selected portions of data in a storage management system

• US 9,588,849 B2
• Filed: 01/20/2015
• Issued: 03/07/2017
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Sinha, Prosenjit
• Primary Examiner(s): Faal, Baboucarr
• Application Number:
  US14/601,053
• Publication Number:
  US 20160210202A1
• Time in Patent Office: 777 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Disclosed methods and systems leverage resources in a storage management system to partially synchronize primary data files based on synchronizing selected portions thereof without regard to changes that may be occurring in other non-synchronized portions. Accordingly, a number of primary data files may be partially synchronized by synchronizing designated portions thereof via auto-restore operations from backup data. This approach relies on storage management resources to designate portions of source data that is to be kept synchronized across any number of targets; detect changes to the designated portions; back up changes to secondary storage; and distribute the changes from secondary storage to the associated targets, with minimal impact to the primary data environment. The approach may be mutually applied, so that changes in any one of an associated group of source data files may be likewise detected, backed up, and distributed to the other members of the group.

Claims

1. A storage management system for partial synchronization of primary data files based on backup and restore operations of portions thereof, the storage management system comprising:

a first client computing device comprising a first data agent;

a first primary storage device in communication with the first client computing device, wherein the first primary storage device comprises a first primary data file generated by the first client computing device, and wherein the first primary data file comprises a first portion;

a second client computing device comprising a second data agent;

a second primary storage device in communication with the second client computing device, wherein the second primary storage device comprises a second primary data file generated by the second client computing device, and wherein the second primary data file comprises a second portion;

a secondary storage computing device comprising a media agent;

a secondary storage device in communication with the secondary storage computing device;

a storage manager in communication with the first client computing device, the second client computing device, and the secondary storage computing device, wherein the storage manager comprises metadata for partially synchronizing the first primary data file and the second primary data file in the storage management system;

wherein the first data agent is configured to;

receive, from the storage manager, first metadata that identifies the first portion of the first primary data file,

detect at least one change in the first portion of the first primary data file, and

back up, in conjunction with the media agent, the first portion of the first primary data file to a secondary copy of the first portion,

wherein the secondary copy of the first portion is stored to the secondary storage device in a backup format; and

wherein the second data agent is configured to;

restore, in conjunction with the media agent,

the secondary copy of the first portion from the secondary storage device to the second portion of the second primary data file on the second primary storage device,

thereby transforming the secondary copy of the first portion from the backup format to a native format accessible as primary data to the second client computing device;

wherein after the backup and restore operations, the first portion of the first primary data file and the second portion of the second primary data file are synchronized with each other without the rest of the first primary data file and the rest of the second primary data file being respectively synchronized with each other,

thereby the first primary data file and the second primary data file being partially synchronized based on backup and restore operations in the storage management system.

2. The storage management system of claim 1 wherein the storage manager is configured to:

instruct the first data agent and the media agent to back up the first portion of the first primary data file to the secondary copy, and

instruct the second data agent and the media agent, after the backup of the first portion is complete, to restore the secondary copy to the second portion of the second primary data file.

3. The storage management system of claim 2 wherein the storage manager is further configured to instruct the first data agent and the media agent to back up the first portion of the first primary data file based on a notice of a detected change to the first portion of the first primary data file received by the storage manager from the first data agent.

4. The storage management system of claim 2 wherein the storage manager is further configured to:

store first metadata that identifies the first portion of the first primary data file, second metadata that identifies the second portion of the second primary data file, and an association between the first portion and the second portion,

transmit the first metadata to the first data agent, and

transmit the second metadata to the second data agent.

5. The storage management system of claim 1 wherein the storage manager is configured to manage a partial-synchronization operation for the first primary data file and the second primary data file, wherein the storage manager is further configured to:

instruct the first data agent and the media agent to back up the first portion of the first primary data file to the secondary copy, and

instruct the second data agent and the media agent, after the backup of the first portion is complete, to restore the secondary copy to the second portion of the second primary data file.

6. The storage management system of claim 1 wherein the first data agent is further configured to detect at least one change in the first portion of the first primary data file based on an operating-system change log in the first client computing device.

7. The storage management system of claim 1 wherein the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively, regardless of any changes to other portions of the first primary data file that fall outside the first portion, and also regardless of any changes to other portions of the second primary data file that fall outside the second portion.

8. The storage management system of claim 1 wherein the second data agent is further configured to:

detect at least one change in the second portion of the second primary data file, and

back up, in conjunction with the media agent, the second portion of the second primary data file to a secondary copy of the second portion, wherein the secondary copy of the secondary portion is stored to the secondary storage device in the backup format; and

wherein the first data agent is further configured to;

restore, in conjunction with the media agent,

the secondary copy of the second portion to the first portion of the first primary data file, thereby transforming the secondary copy of the second portion from the backup format to a native format accessible as primary data to the first client computing device.

9. A storage management system for partial synchronization of primary data files based on backup and restore operations of portions thereof, the storage management system comprising:

a first client computing device comprising a first data agent;

a first primary storage device in communication with the first client computing device, wherein the first primary storage device comprises a first primary data file generated by the first client computing device, and wherein the first primary data file comprises a first portion;

a second client computing device comprising a second data agent;

a second primary storage device in communication with the second client computing device, wherein the second primary storage device comprises a second primary data file generated by the second client computing device, and wherein the second primary data file comprises a second portion;

a secondary storage computing device comprising a media agent;

a secondary storage device in communication with the secondary storage computing device;

a storage manager in communication with the first client computing device, the second client computing device, and the secondary storage computing device, wherein the storage manager comprises metadata for partial-synchronization operations in the storage management system;

wherein to manage a partial-synchronization operation for the first primary data file and the second primary data file, the storage manager is configured to;

based on a notice of a detected change to the first portion of the first primary data file received from the first data agent, instruct the first data agent and the media agent to back up the first portion of the first primary data file to a secondary copy of the first portion, which is stored to the secondary storage device in a backup format, and

instruct the second data agent and the media agent, after the backup of the first portion is complete, to restore the secondary copy of the first portion from the secondary storage device to the second portion of the second primary data file in the second primary storage device,

thereby transforming the secondary copy of the first portion from the backup format to a native format accessible as primary data to the second client computing device;

wherein after the backup and restore operations, the first portion of the first primary data file and the second portion of the second primary data file are synchronized with each other without the rest of the first primary data file and the rest of the second primary data file being respectively synchronized with each other.

10. The storage management system of claim 9 wherein the instruction to restore the secondary copy of the first portion of the first primary data file to the second portion of the second primary data file is based on an association between the first portion and the second portion that is part of the metadata in the storage manager.

11. The storage management system of claim 9 wherein the storage manager is further configured to:

store first metadata that identifies the first portion of the first primary data file, second metadata that identifies the second portion of the second primary data file, and an association between the first portion and the second portion,

transmit the first metadata to the first data agent, and

transmit the second metadata to the second data agent.

12. The storage management system of claim 9 wherein the storage manager is further configured to:

store first metadata that identifies the first portion of the first primary data file, second metadata that identifies the second portion of the second primary data file, and an association between the first portion and the second portion,

transmit the first metadata to the first data agent, and

transmit the second metadata to the second data agent;

wherein the storage manager is configured to instruct the second data agent to restore the secondary copy of the first portion of the first primary data file to the second portion of the second primary data file based on the association between the first portion and the second portion.

13. The storage management system of claim 9 wherein the storage manager is further configured to:

store first metadata that identifies the first portion of the first primary data file, second metadata that identifies the second portion of the second primary data file, and an association between the first portion and the second portion,

transmit the first metadata to the first data agent,

instruct the first data agent to monitor changes to the first portion of the first primary data file,

transmit the second metadata to the second data agent, and

instruct the second data agent to monitor changes to the second portion of the second primary data file.

14. The storage management system of claim 9 wherein after the partial-synchronization operation managed by the storage manager, the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively,

regardless of any changes to other portions of the first primary data file that fall outside the first portion, and

also regardless of any changes to other portions of the second primary data file that fall outside the second portion.

15. A method for partially synchronizing primary data files based on backup and restore operations of portions thereof in a storage management system, the method comprising:

storing, by a storage manager, first metadata that identifies a first portion of a first primary data file, second metadata that identifies a second portion of a second primary data file, and an association between the first portion and the second portion;

managing, by the storage manager, a partial-synchronization operation for the first primary data file and the second primary data file relative to the first portion and the second portion, respectively, based at least in part on the association between the first portion and the second portion,

wherein the managing of the partial-synchronization operation is based on a notice of a detected change to the first portion of the first primary data file received from a first data agent executing in the storage management system, and wherein the managing of the partial-synchronization operation comprises;

instructing the first data agent and a media agent executing in the storage management system to back up the first portion of the first primary data file to a secondary copy of the first portion, wherein the secondary copy of the first portion is stored in a backup format to a secondary storage device in communication with the media agent, and

after the backup of the first portion is complete, instructing the media agent and a second data agent that is associated with the second primary data file to restore the secondary copy of the first portion from the secondary storage device to the second portion of the second primary data file,

thereby transforming the secondary copy of the first portion from the backup format to a native format accessible as primary data to an application associated with the second data agent;

wherein after the backup and restore operations are complete, the first portion of the first primary data file and the second portion of the second primary data file are synchronized with each other without the rest of the first primary data file and the rest of the second primary data file being respectively synchronized with each other.

16. The method of claim 15 further comprising:

transmitting, by the storage manager, the first metadata to the first data agent; and

transmitting, by the storage manager, the second metadata to the second data agent.

17. The method of claim 15 further comprising:

transmitting, by the storage manager, the first metadata to the first data agent;

instructing, by the storage manager, the first data agent to monitor changes to the first portion of the first primary data file;

transmitting, by the storage manager, the second metadata to the second data agent; and

instructing, by the storage manager, the second data agent to monitor changes to the second portion of the second primary data file.

18. The method of claim 15 wherein after the partial-synchronization operation managed by the storage manager, the first primary data file and the second primary data file are partially synchronized relative to the first portion and the second portion, respectively,

regardless of any changes to other portions of the first primary data file that fall outside the first portion, and

also regardless of any changes to other portions of the second primary data file that fall outside the second portion.

19. The method of claim 15 further comprising:

receiving, by the first data agent executing on a first client computing device in the storage management system, the first metadata that identifies the first portion of the first primary data file, wherein the first primary data file comprises data generated by an application associated with the first data agent that also executes on the first client computing device;

detecting, by the first data agent, at least one change in the first portion of the first primary data file;

transmitting, by the first data agent to a storage manager, a notice of the detected at least one change in the first portion of the first primary data file; and

executing, by the first data agent, at least part of the backup operation of the first portion of the first primary data file to the secondary copy of the first portion of the primary data file.

20. The method of claim 19 further comprising:

executing, by the second data agent, which executes on a second client computing device in the storage management system, at least part of the restore operation of the secondary copy to the second portion of a second primary data file.