Method and system for performing periodic replication using a log

• US 7,406,487 B1
• Filed: 08/29/2003
• Issued: 07/29/2008
• Est. Priority Date: 08/29/2003
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (NortonLifeLock Inc.)
• Inventors: Krishnamurthy, Raghu, Karmarkar, Kedar M., Gupta, Vikas K.
• Primary Examiner(s): Fleurantin; Jean B
• Application Number:
  US10/652,380
• Time in Patent Office: 1,796 Days
• Field of Search: 707 1- 10, 707200-206, 707100-1041, 711/114, 711155-156, 711161-165, 711/171, 714/6
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

Disclosed is a method and system for performing periodic replication using a write-ordered log. According to one embodiment, a plurality of write operations to a primary data volume are tracked using a write operation log and then data associated with the plurality of write operations is replicated to a secondary data volume by coalescing the plurality of write operations utilizing the write operation log and transferring data associated with the plurality of write operations to the secondary data volume. According to another embodiment the described tracking includes storing metadata associated with the plurality of write operations within the write operation log. In another embodiment, the described coalescing includes identifying a non-overlapping portion of a first write operation and a second write operation of the plurality of write operations utilizing the metadata.

Claims

1. A method implemented by one or more computers, the method comprising:

storing metadata for each of a plurality of write operations to a primary data volume within a log;

wherein the plurality of write operations comprises first and second write operations comprising first and second data, respectively, wherein the metadata stored in the log for the first write operation comprises first information, wherein the first information defines a first extent of the primary data volume where the first data is written or is to be written and wherein the metadata stored in the log for the second write operation comprises second information, wherein the second information defines a second extent of the primary data volume where the second data is written or is to be written;

comparing the first extent with the second extent to identify a portion(s) of the second extent that does not overlap with the first extent;

replicating the first data and a portion of the second data that is written or is to be written to the portion(s) of the second extent that does not overlap with the first extent;

wherein the first information comprises (1) a first offset from a beginning of the primary data volume, and (2) a first size data that defines the length of the first extent, and wherein the second information comprises (1) a second offset from the beginning of the primary data volume, and (2) a second size data that defines the length of the second extent.

2. The method of claim 1, further comprising:

storing the first and second data written by each of said first and second write operations within said log.

3. The method of claim 1, further comprising:

storing the first or second data within a point-in-time copy of said primary data volume.

4. The method of claim 1, wherein said storing metadata comprises,

storing metadata for each of said plurality of write operations in a first order within said log.

5. The method of claim 4, wherein storing metadata for each of said plurality of write operations in a first order within said log comprises:

storing metadata for each of said plurality of write operations in write order within said log.

6. The method of claim 4 further comprising reading metadata from said log in a second order, wherein said second order is a reverse of said first order.

7. The method of claim 6 further comprising storing a marker within said log.

8. The method of claim 1, wherein said replicating comprises writing the first data and the portion of the second data that is written or is to be written to the portion(s) of the second extent that does not overlap with the first extent to a secondary data volume.

9. An apparatus comprising:

means for storing metadata for each of a plurality of write operations to a primary data volume within a log;

wherein the plurality of write operations comprises first and second write operations comprising first and second data, respectively, wherein the metadata stored in the log for the first write operation comprises first information, wherein the first information defines a first extent of the primary data volume where the first data is written or is to be written and wherein the metadata stored in the log for the second write operation comprises second information, wherein the second information defines a second extent of the primary data volume where the second data is written or is to be written;

means for comparing the first extent with the second extent to identify a portion(s) of the second extent that does not overlap with the first extent;

means for replicating the first data and a portion of the second data that is written or is to be written to the portion(s) of the second extent that does not overlap with the first extent;

wherein the first information comprises (1) a first offset from a beginning of the primary data volume, and (2) a first size data that defines the length of the first extent, and wherein the second information comprises (1) a second offset from the beginning of the primary data volume, and (2) a second size data that defines the length of the second extent.

10. The apparatus of claim 9, further comprising:

means for storing first and second data within said log.

11. The apparatus of claim 9, wherein,

said means for storing metadata comprises means for storing metadata for each of said plurality of write operations in a first logical order within said log.

12. A machine-readable storage medium having a plurality of instructions executable by a machine embodied therein, wherein said plurality of instructions, when executed, cause said machine to perform a method comprising:

storing metadata for each of a plurality of write operations to a primary data volume within a log;

wherein the plurality of write operations comprises first and second write operations comprising first and second data, respectively, wherein the metadata stored in the log for the first write operation comprises first information, wherein the first information defines a first extent of the primary data volume where the first data is written or is to be written and wherein the metadata stored in the log for the second write operation comprises second information, wherein the second information defines a second extent of the primary data volume where the second data is written or is to be written;

comparing the first extent with the second extent to identify a portion(s) of the second extent that does not overlap with the first extent;

replicating the first data and a portion of the second data that is written or is to be written to the portion(s) of the second extent that does not overlap with the first extent;

wherein the first information comprises (1) a first offset from a beginning of the primary data volume, and (2) a first size data that defines the length of the first extent, and wherein the second information comprises (1) a second offset from the beginning of the primary data volume, and (2) a second size data that defines the length of the second extent.

13. The machine-readable storage medium of claim 12, said method further comprising:

storing the first and second data within said log.

14. The machine-readable storage medium of claim 12, wherein,

said storing metadata comprises storing metadata for each of said plurality of write operations in a first logical order within said log.

15. A system comprising:

a memory and a processor;

a primary data volume; and

a log for storing metadata for first and second write operations comprising first and second data, respectively, wherein the metadata for the first write operation comprises first information, wherein the first information defines a first extent of the primary data volume where the first data is written or is to be written and wherein the metadata for the second write operation comprises second information, wherein the second information defines a second extent of the primary data volume where the second data is written or is to be written;

a circuit for comparing the first extent with the second extent to identify a portion(s) of the second extent that does not overlap with the first extent;

a circuit for replicating the first data and a portion of the second data that is written or is to be written to the portion(s) of the second extent that does not overlap with the first extent;

wherein the first information comprises (1) a first offset from a beginning of the primary data volume, and (2) a first size data that defines the length of the first extent, and wherein the second information comprises (1) a second offset from the beginning of the primary data volume, and (2) a second size data that defines the length of the second extent.