Systems and methods for resynchronizing information

• US 9,002,799 B2
• Filed: 02/14/2014
• Issued: 04/07/2015
• Est. Priority Date: 12/19/2005
• Status: Expired due to Fees

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David, Agrawal, Vijay
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US14/181,359
• Publication Number:
  US 20140164327A1
• Time in Patent Office: 417 Days
• Field of Search: 707/655
• US Class Current: 707/655
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/1787 Details of non-transparently synchronising file systems
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods and systems for synchronizing data files in a storage network between a first and a second storage device is provided. The method includes storing first data files associated with the first storage device to a storage medium, whereby the first data files include first data records. The storage medium may then be transferred to the second storage device. The first data files from the storage medium may be loaded onto the second storage device. The second data records from the first storage device may be received, and the first and second data records are compared. The first data files at the second storage device may be updated based on the comparison of the first and second data records.

Claims

1. A system of synchronizing data between a first and second storage device comprising:

a first log stored in computer readable memory, the first log captures changes to first data stored on a first storage device, the first log comprising at least a first set of changes to first data stored on at least a first storage device;

a first replication manager comprising computer hardware that transmits over a network at least a portion of the first set of changes captured in the first log to a second log stored on one or more second storage devices, wherein transmitting the portion of the first set of changes creates a second set of changes in the second log;

a second replication manager comprising computer hardware that replicates the first set of changes to the first data, by performing the second set of changes in the second log to create second data stored on the one or more second storage devices, wherein the second data is a replication of the first data stored on the first storage device;

after detection of a network communication error, at least one of the first and second replication managers compare the second set of changes in the second log with the first set of changes to the first data to determine whether there is missing change data in the second set of changes; and

at least one of the first and second replication managers update the second data on the one or more second storage devices upon detection of the missing change data, wherein the update of the second data comprises copying at least a portion of the first data stored on the first storage device to the one or more second storage devices.

2. The system of claim 1 wherein the record identifiers are associated with change journal entries.

3. The system of claim 1 wherein the record identifiers comprise at least an update sequence number.

4. The system of claim 1 wherein the record identifiers comprise at least a file reference number.

5. A method of replicating data comprising:

storing in a first log, changes to first data stored on one or more first storage devices, the first log further comprises record identifiers that are associated with the changes to the data files;

copying at least a portion of a first set of changes and a first set of record identifiers in the first log to a second log stored on one or more second storage devices to create a second set of changes and a second set of record identifiers in the second log;

replicating with computer hardware, the portion of the first set of changes to the first data, by performing the second set of changes in the second log to create second data stored on the one or more second storage devices, wherein the second data is a replication of the first data stored on the one or more first storage devices;

comparing with computer hardware, the second set of record identifiers in the second log with the first set of record identifiers in the first log to determine whether there is missing change data in the second set of changes; and

updating with computer hardware, the second data on the one or more second storage devices upon detection of the missing change data, wherein updating the second data comprises copying at least a portion of the first data stored on the one or more first storage devices to the one or more second storage devices.

6. The method of claim 5 wherein copying the portion of the first set of changes and the first set of record identifiers in the first log to the second log the transmitting occurs upon reaching a threshold capacity of the first log.

7. The method of claim 5 further comprising associating the record identifiers with change journal entries.

8. The method of claim 7 further comprising comparing at least one of the record identifiers associated with the change journal entries to a record identifier of a previously stored change journal entry on the second storage device, wherein non-identical identifiers signify non-synchronized data.

9. The method of claim 5 wherein the record identifiers comprise at least an update sequence number.

10. The method of claim 5 wherein the record identifiers comprise at least a file reference number.

11. The method of claim 5 wherein the record identifiers comprise at least one of (i) a time stamp, and (ii) information about a type of change.

12. The method of claim 5 further comprising resynchronizing the replication based on missing record identifiers.

13. A data replication system comprising:

a first log stored in computer accessible memory, the first log records changes to first data stored on one or more first storage devices, the first log further comprising record identifiers that are associated with the changes to the data files;

a first replication manager comprising computer hardware that copies at least a portion of a first set of changes and a first set of record identifiers in the first log to a second log stored on one or more second storage devices to create a second set of changes a second set of record identifiers in the second change log;

a second replication manager comprising computer hardware that replicates at least a portion of the first set of changes to the first data, by performing the second set of changes in the second log to create second data stored on the one or more second storage devices, wherein the second data is a replication of the first data stored on the one or more first storage devices;

at least one of the first and second replication managers compares the second set of record identifiers in the second log with the first set of record identifiers in the first log to determine whether there is missing change data in the second set of changes; and

at least one of the first and second replication managers updates the second data on the one or more second storage devices upon detection of the missing change data, wherein updating the second data comprises copying at least a portion of the first data stored on the one or more first storage devices to the one or more second storage devices.

14. The data replication system of claim 13 wherein the computer hardware copies the portion of the first set of changes and the first set of record identifiers in the first log to the second log upon reaching a threshold capacity of the first log.

15. The data replication system of claim 13 wherein the record identifiers are associated with change journal entries.

16. The data replication system of claim 15 further comprising comparing at least one of the record identifiers associated with the change journal entries to a record identifier of a previously stored change journal entry on the second storage device, wherein non-identical identifiers signify non-synchronized data.

17. The data replication system of claim 13 wherein the record identifiers comprise at least an update sequence number.

18. The data replication system of claim 13 wherein the record identifiers comprise at least a file reference number.

19. The data replication system of claim 13 wherein the record identifiers comprise at least a file reference number.

20. The data replication system of claim 13 wherein the record identifiers comprise at least one of (i) a time stamp, and (ii) information about a type of change.