Coordinated dirty block tracking

• US 7,039,661 B1
• Filed: 12/29/2003
• Issued: 05/02/2006
• Est. Priority Date: 12/29/2003
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Veritas Operating Corporation (Gen Digital Inc.)
• Inventors: Ranade, Dilip M.
• Primary Examiner(s): ALAM, SHAHID AL
• Application Number:
  US10/747,782
• Time in Patent Office: 855 Days
• Field of Search: 707/10, 707/100, 707/104.1, 707/204, 707/3, 707/203, 707/205, 707/206, 707/201, 707/202, 709/219, 709/223, 711/162, 711/202, 714/5, 714/6, 714/15
• US Class Current: 707/610
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99954 Version management
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method, system, computer system, and computer-readable medium to maintain consistent, up-to-date copies of primary data that enable quick resumption of operations upon failure of the primary node, the secondary node, and/or the network connecting nodes. The current status of individual storage locations is tracked, indicating whether data in each individual storage location is synchronized between a primary node and a secondary node. Transaction logging by a transaction-logging application, such as a file system or database system, is used to track individual storage locations updated with each transaction. Acknowledgements of these updates are tracked during replication to determine whether individual storage locations containing primary and secondary data are synchronized. This coordinated transaction logging and acknowledgement tracking enables write operations and replication to be performed efficiently while maintaining the information necessary to resynchronize only individual storage locations when primary and secondary data become unsynchronized.

Claims

1. A computer-implemented method comprising:

in response to obtaining first data for replication to a secondary storage area, causing an indicator to be set for a first location, wherein

the first data are stored in the first location of a first storage area, and

the indicator indicates that the first data are unsynchronized with corresponding data in a corresponding location of the secondary storage area; and

causing an entry to be added to a list of entries for the first location, wherein each entry in the list is related to an update to data in the first location, and the list can be used to reset the indicator for the first location;

writing a record to a transaction log, wherein the record comprises the first location;

in response to receiving an acknowledgement that the first data have been received at the secondary storage area,

causing the first location to be removed from each active transaction of at least one active transaction in the transaction log; and

if one active transaction of the at least one active transaction in the transaction log is completed,

writing a second record to the transaction log indicating that the one active transaction is complete.

2. The method of claim 1 further comprising:

adding an update identifier to the entry.

3. The method of claim 1 further comprising:

in response to receiving the acknowledgement that the first data have been received at the secondary storage area,

causing the entry to be removed from the list of entries for the first location.

4. The method of claim 1, further comprising:

in response to the writing the second record to the transaction log, freeing space in the transaction log previously used by the one active transaction.

5. The method of claim 3 further comprising:

in response to removing the entry from the list of entries for the first location, determining whether the list of entries for the first location is empty; and

if the list of entries for the first location is empty, resetting the indicator for the first location.

6. The method of claim 3 further comprising:

in response to removing the entry from the list of entries for the first location, determining whether the list of entries for the first location is empty, and

if the list of entries for the first location is empty, causing the indicator for the first location to be reset.

7. The method of claim 3 wherein

the causing the entry to be removed from the list of entries for the first location comprises removing the entry from the list of entries for the first location.

8. The method of claim 1 wherein

the causing the indicator to be set comprises setting the indicator.

9. The method of claim 1 wherein

the causing the entry to be added to the list of entries for the first location comprises adding the entry to the list of entries for the first location.

10. The method of claim 1 wherein

if the acknowledgement that the first data have been received at the secondary storage area is not received,

causing the indicator to be written to a persistent storage area.

11. The method of claim 10 further comprising:

using the indicator to synchronize the first data in the first location of the first storage area with the corresponding data in the corresponding location of the secondary storage area.

24. The method of claim 1, wherein

a file system sets the indicator for the first location;

and

the file system adds the entry to the list of entries for the first location.

25. The method of claim 3, wherein

a replicator receives the acknowledgement;

and

the replicator causes a file system to remove the entry from the list of entries for the first location.

26. The method of claim 1, wherein

a replicator receives the acknowledgement;

the replicator causes a file system to remove the first location; and

the file system writes the second record indicating that the one active transaction is completed.

12. A system comprising:

causing means for causing an indicator to be set for a first location in response to obtaining first data for replication to a secondary storage area, wherein

the first data are stored in the first location of a first storage area, and

the indicator indicates that the first data are unsynchronized with corresponding data in a corresponding location of the secondary storage area;

second causing means for causing an entry to be added to a list of entries for the first location, wherein

each entry in the list is related to an update to data in the first location, and

the list can be used to reset the indicator for the first location;

writing means for writing a record to a transaction log, wherein the record comprises the first location;

third causing means for causing the first location to be removed from each active transaction of at least one active transaction in the transaction log in response to receiving an acknowledgement that the first data have been received at the secondary storage area; and

second writing means for writing a second record to the transaction log indicating that one active transaction is complete if the one active transaction of the at least one active transaction in the log is completed.

13. The system of claim 12 further comprising:

fourth causing means for causing the entry to be removed from the list of entries for the first location in response to receiving the acknowledgement that the first data have been received at the secondary storage area.

14. The system of claim 13 further comprising:

determining means for determining whether the list of entries for the first location is empty in response to removing the entry from the list of entries for the first location; and

resetting means for resetting the indicator for the first location if the list of entries for the first location is empty.

28. The system of claim 12, wherein

a file system sets the indicator for the first location;

and

the file system adds the entry to the list of entries for the first location.

29. The system of claim 13, wherein

a replicator receives the acknowledgement;

and

the replicator causes a file system to remove the entry from the list of entries for the first location.

30. The system of claim 12, wherein

a replicator receives the acknowledgement;

the replicator causes a file system to remove the first location; and

the file system writes the second record indicating that the one active transaction is completed.

15. A computer-readable medium comprising:

causing instructions to cause an indicator to be set for a first location in response to obtaining first data for replication to a secondary storage area, wherein

the first data are stored in the first location of a first storage area, and

the indicator indicates that the first data are unsynchronized with corresponding data in a corresponding location of the secondary storage area, and

second causing instructions to cause an entry to be added to a list of entries for the first location, wherein

each entry in the list is related to an update to data in the first location, and

the list can be used to reset the indicator for the first location;

writing instructions to write a record to a transaction log, wherein the record comprises the first location;

third causing instructions to cause the first location to be removed from each active transaction of at least one active transaction in the transaction log in response to receiving an acknowledgement that the first data have been received at the secondary storage area; and

second writing instructions to write a second record to the transaction log, wherein the second record indicates that one active transaction of the at least one active transaction in the transaction log is completed.

16. The computer-readable medium of claim 15 further comprising:

fourth causing instructions to cause an update identifier to be added to the entry.

17. The computer-readable medium of claim 15 further comprising:

fourth causing instructions to cause the entry to be removed from the list of entries for the first location in response to receiving the acknowledgement that the first data have been received at the secondary storage area.

18. The computer-readable medium of claim 17 further comprising:

determining instructions to determine whether the list of entries for the first location is empty in response to removing the entry from the list of entries for the first location; and

resetting instructions to reset the indicator for the first location if the list of entries for the first location is empty.

19. A computer system comprising:

a processor, and

the computer-readable medium of claim 15, wherein the computer-readable medium is coupled to the processor.

32. The computer-readable medium of claim 15, wherein

a file system sets the indicator for the first location;

and

the file system adds the entry to the list of entries for the first location.

33. The computer-readable medium of claim 17, wherein

a replicator receives the acknowledgement;

and

the replicator causes a file system to remove the entry from the list of entries for the first location.

34. The computer-readable medium of claim 15, wherein

a replicator receives the acknowledgement;

the replicator causes a file system to remove the first location; and

the file system writes the second record indicating that the one active transaction is completed.

35. The computer-readable medium of claim 18, wherein the file system resets the indicator.

20. A system comprising:

a processor; and

a memory, wherein the memory comprises;

a causing module to cause an indicator to be set for a first location in response to obtaining first data for replication to a secondary storage area, wherein the first data are stored in the first location of a first storage area, and

the indicator indicates that the first data are unsynchronized with corresponding data in a corresponding location of the secondary storage area, and

a second causing module to cause an entry to be added to a list of entries for the first location, wherein

each entry in the list is related to an update to data in the first location, and

the list can be used to reset the indicator for the first location;

a writing module to write a record to a transaction log, wherein the record comprises the first location;

a third causing module to cause the first location to be removed from each active transaction of at least one active transaction in the transaction log in response to receiving an acknowledgement that the first data have been received at the secondary storage area; and

a second writing module to write a record to the transaction log indicating that one active transaction is complete if the one active transaction of the at least one active transaction in the transaction log is completed.

21. The system of claim 20 further comprising:

a fourth causing module to cause an update identifier to be added to the entry.

22. The system of claim 20 further comprising:

a fourth causing module to cause the entry to be removed from the list of entries for the first location in response to receiving the acknowledgement that the first data have been received at the secondary storage area.

23. The system of claim 22 further comprising:

a determining module to determine whether the list of entries for the first location is empty in response to removing the entry from the list of entries for the first location; and

a resetting module to reset the indicator for the first location if the list of entries for the first location is empty.

36. The system of claim 20, wherein

a file system sets the indicator for the first location;

and

the file system adds the entry to the list of entries for the first location.

37. The system of claim 22, wherein

a replicator receives the acknowledgement;

and

the replicator causes a file system to remove the entry from the list of entries for the first location.

38. The system of claim 20, wherein

a replicator receives the acknowledgement;

the replicator causes a file system to remove the first location; and

the file system writes the second record indicating that the one active transaction is completed.

39. The system of claim 23, wherein the file system resets the indicator.

27. A method comprising:

causing an indicator to be set for a first location in response to obtaining first data for replication to a secondary storage area, wherein

the first data are stored in the first location of a first storage area, and

the indicator indicates that the first data are unsynchronized with corresponding data in a corresponding location of the secondary storage area;

causing an entry to be added to a list of entries for the first location, wherein each entry in the list is related to an update to data in the first location, and

the list can be used to reset the indicator for the first location;

determining whether the list of entries for the first location is empty in response to removing the entry from the list of entries for the first location; and

resetting the indicator for the first location if the list of entries for the first location is empty, wherein

the file system resets the indicator.

31. A system comprising:

causing means for causing an indicator to be set for a first location in response to obtaining first data for replication to a secondary storage area, wherein the first data are stored in the first location of a first storage area, and

the indicator indicates that the first data are unsynchronized with corresponding data in a corresponding location of the secondary storage area;

second causing means for causing an entry to be added to a list of entries for the first location, wherein

each entry in the list is related to an update to data in the first location, and

the list can be used to reset the indicator for the first location;

determining means for determining whether the list of entries for the first location is empty in response to removing the entry from the list of entries for the first location; and

resetting means for resetting the indicator for the first location if the list of entries for the first location is empty, wherein

the file system resets the indicator.