Systems and methods for resynchronizing information

• US 20070198602A1
• Filed: 12/15/2006
• Published: 08/23/2007
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Ngo, David, Agrawal, Vijay
• Granted Patent: US 8,655,850 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/1787 Details of non-transparently synchronising file systems
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods and systems for synchronizing data files in a storage network between a first and a second storage device is provided. The method includes storing first data files associated with the first storage device to a storage medium, whereby the first data files include first data records. The storage medium may then be transferred to the second storage device. The first data files from the storage medium may be loaded onto the second storage device. The second data records from the first storage device may be received, and the first and second data records are compared. The first data files at the second storage device may be updated based on the comparison of the first and second data records.

Claims

1. A method of synchronizing data between a first and second storage device comprising:

identifying a first set of data on a first storage device for replication;

capturing the first set of data in an a first log;

determining changes to the first set of data;

recording a second set data in a change log, the second set of data describing the changes to the first set of data;

transmitting the first and second set of data to the second storage device; and

replicating the changes in the first set of data on the second storage device using the second set of data.

2. The method of claim 1 wherein the transmitting the first and second set of data is done periodically.

3. The method of claim 1 wherein the step of transmitting is executed upon reaching a threshold capacity of the first log.

4. The method of claim 1 further comprising assigning an identifier to a change log entry.

5. The method of claim 4 further comprising comparing the identifier of the change journal entry to an identifier of a previously stored change journal entry on the second storage device, wherein non-identical identifiers signify non-synchronized data.

6. The method of claim 4 wherein the identifier includes an update sequence number.

7. The method of claim 6 wherein a highest update sequence number indicates the most recent change data.

8. The method of claim 4 wherein the identifier includes a file reference number.

9. The method of claim 1 wherein the second set of data is metadata.

10. The method of claim 1 further comprising storing the second set of data in a second log, the second log linked to the second storage device.

11. A method of synchronizing data after an interruption of data transfer between a first and a second storage device, the method comprising:

detecting an interruption in the data transfer between the first and the second storage device;

comparing first logged data records in a first data log associated with the first storage device with second logged records in a second data log associated with the second storage device; and

sending updated data files from the first storage device to the second storage device based on comparing the first and the second logged records.

12. The method of claim 11 further comprising comparing record identifier numbers in the first logged data records with record identifier numbers in the second logged data records.

13. The method of claim 12 further comprising identifying the updated data files of the first storage device using the record identifier numbers.

14. The method of claim 12 wherein the record identifier numbers of the first and second logged data records include an update sequence number.

15. The method of claim 11 wherein the updated data files include a highest update sequence number.

16. The method of claim 12 wherein the record identifier numbers of the first and second logged data records include a file reference number.

17. A method of replicating data on an electronic storage system network comprising:

storing a set of data on a first storage device, the set of data including a record identifier;

copying the set of data to an intermediary storage device;

transferring the set of data from the intermediary storage device to a second storage device;

comparing the record identifiers of the set of data on the third storage device to the record identifier of the set of data on the first storage device; and

updating the set of data on the third storage device upon detection of non-identical record identifiers, the updated data transmitted across the network.

18. The method of claim 17 wherein the record identifiers include an update sequence number.

19. The method of claim 18 wherein the updated data includes a highest update sequence number.

20. The method of claim 17 wherein the record identifiers include a file reference number.

21. A system for replicating data on an electronic storage network comprising:

a first and second storage device;

a first log stored on the first storage device, the first log tracking changes to data stored on the first storage device; and

a replication manager module in communication with the first and second storage devices, the replication manager module transmitting updated data from the first log to the second storage device.

22. The system of claim 21 further comprising a second log stored on the second storage device, the second log storing updated data records transmitted from the replication manager module.

23. The system of claim 21 wherein the data and updated data records include record identifiers, the record identifiers compared between the data stored on the first storage device and the updated data records stored on the second storage device.

24. The system of claim 23 wherein non-identical record identifiers between the data stored on the first storage device and the updated data records stored on the second storage device signify non-synchronized data.

25. The system of claim 23 wherein the record identifier includes an update sequence number.

26. The system of claim 25 wherein a highest update sequence number signifies the most current data.

27. The system of claim 23 wherein the record identifier includes a file reference number.

28. The system of claim 21 further comprising a log driver, the driver storing updated data to the first log.

29. The system of claim 21 further comprising a change journal, the change journal including metadata corresponding to the updated data.

30. The system of claim 29 further comprising a change journal driver, the change journal driver storing updated metadata to the change journal.

31. The system of claim 21 wherein the replication manager module transmits updated data to the second storage device periodically.

32. The system of claim 21 wherein the replication manager module transmits updated data to the second storage device upon the first log reaching a threshold capacity.

33. A computer-readable medium having stored thereon a plurality of sequences of instructions which, when executed by one or more processors cause an electronic device to:

store in a first log changes to data on a first storage device, the log including record identifiers;

transmit updated data from the first log to a second log on a second storage device;

compare the record identifier of the data from the first log to the record identifier of the data from the second log; and

update the second storage device with the updated data upon detecting a difference in the record identifiers.

34. A computer-readable medium having stored thereon a plurality of sequences of instructions which, when executed by one or more processors cause an electronic device to:

detect a failure event in a data replication operation between first and second storage devices;

store updates of a first set of data in the first storage device;

log a second set of data detailing the updates to the first set of data, the second set of data including record identifiers;

compare the record identifiers from the first storage device to a record identifier of the second storage device; and

replicate the updates to the first set of data on the second storage device, the second set of data identifying the updates.