Efficient logging for asynchronously replicating volume groups

• US 8,285,956 B2
• Filed: 10/22/2009
• Issued: 10/09/2012
• Est. Priority Date: 10/22/2009
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Ozdemir, Kadir
• Primary Examiner(s): Chery, Mardochee
• Application Number:
  US12/603,673
• Publication Number:
  US 20110099342A1
• Time in Patent Office: 1,083 Days
• Field of Search: 711/162, 711/E12.002, 711/E12.103
• US Class Current: 711/162
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2074 Asynchronous techniques

Abstract

A system and method for logging for asynchronously replicating volume groups. A write request to write data to a location in a volume may be received. Metadata associated with the write request may be stored. It may be determined if the write request possibly overlaps with one or more earlier write requests to the volume that have not yet been replicated to a secondary storage. The data may be stored in a replication log only if the write request possibly overlaps with one or more earlier write requests to the volume. The data may not be stored in the replication log if the write request does not overlap with one or more earlier write requests to the volume. The data may be written to the location in the volume. Changes to the volume may periodically be replicated to the secondary storage using the replication log.

Claims

1. A non-transitory computer-readable storage medium comprising program instructions for replicating data, wherein the program instructions are executable to:

receive a write request to write data to a location in a volume;

store metadata associated with the write request;

maintain information indicating locations of earlier write requests to the volume that have not yet been replicated to a secondary storage;

determine if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage;

store the data in a replication log in response to determining that the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume;

wherein the data is not stored in the replication log in response to determining that the write request does not overlap with one or more locations indicated by the information indicating locations of earlier write requests to the volume;

write the data to the location in the volume in response to the write request; and

periodically replicate changes to the volume to the secondary storage using the replication log.

2. The computer-readable storage medium of claim 1, wherein the program instructions executable to determine if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage are further executable to:

determine if the write request is a first write request that overlaps on a block range of one or more earlier write requests to the volume and that the one or more earlier write requests have not yet been replicated to the secondary storage;

wherein in response to determining that the write request is the first write request that overlaps on the block range, the program instructions are further executable to store data from the volume corresponding to the block range to the replication log.

3. The computer-readable storage medium of claim 2, wherein the program instructions are further executable to:

store linking information, wherein the linking information links an entry in the replication log comprising the data from the volume corresponding to the block range and one or more entries in the replication log corresponding to one or more overlapping earlier write requests.

4. The computer-readable storage medium of claim 1, wherein the program instructions are further executable to:

maintain a data change map data structure in memory, wherein the information indicating locations of earlier write requests to the volume that have not yet been replicated to a secondary storage comprises the data change map data structure;

wherein the program instructions are executable to examine the data change map data structure to determine if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage.

5. The computer-readable storage medium of claim 4,

wherein the data change map data structure comprises a plurality of entries, wherein the plurality of entries track locations of changes to the volume that have not yet been replicated to a secondary storage;

wherein the program instructions for maintaining the data change map data structure in memory are executable to merge entries in the data change map data structure if a number of entries in the data change map data structure reaches a limit.

6. The computer-readable storage medium of claim 1,

wherein, in periodically replicating changes to the volume to the secondary storage using the replication log, the program instructions are executable to:

read a first record from the replication log, wherein the first record refers to a first area of the volume;

determine if at least a portion of the first area of the volume has been overwritten;

if at least a portion of the first area of the volume has not been overwritten, then read first data from the portion of the first area of the volume that has not been overwritten and provide the first data to the secondary storage;

if at least a portion of the first area of the volume has been overwritten, then read second data corresponding to the overwritten portion of the first area of the volume from the replication log and provide the second data to the secondary storage.

7. The computer-readable storage medium of claim 6,

wherein, in determining if at least a portion of the first area of the volume has been overwritten, the program instructions are executable to:

utilize information in one or more data structures to determine if at least a portion of the first area of the volume has possibly been overwritten and to determine one or more portions of the replication log that comprise the second data.

8. A computer-implemented method for replicating data, the method comprising:

a computer system receiving a write request to write data to a location in a volume;

the computer system storing metadata associated with the write request;

the computer system maintaining information indicating locations of earlier write requests to the volume that have not yet been replicated to a secondary storage;

the computer system determining if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage;

the computer system storing the data in a replication log in response to determining that the write request possibly overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume;

wherein the data is not stored in the replication log in response to determining that the write request does not overlap with one or more locations indicated by the information indicating locations of earlier write requests to the volume;

the computer system writing the data to the location in the volume in response to the write request; and

the computer system periodically replicating changes to the volume to the secondary storage using the replication log.

9. The computer-implemented method of claim 8,

wherein said determining comprises determining if the write request is a first write request that overlaps on a block range of one or more earlier write requests to the volume and that the one or more earlier write requests have not yet been replicated to the secondary storage;

wherein in response to determining that the write request is the first write request that overlaps on the block range, the method further comprises the computer system storing data from the volume corresponding to the block range to the replication log.

10. The computer-implemented method of claim 9, further comprising:

the computer system storing linking information, wherein the linking information links an entry in the replication log comprising the data from the volume corresponding to the block range and one or more entries in the replication log corresponding to one or more overlapping earlier write requests.

11. The computer-implemented method of claim 8, further comprising:

the computer system maintaining a data change map data structure in memory, wherein the information indicating locations of earlier write requests to the volume that have not yet been replicated to a secondary storage comprises the data change map data structure;

wherein determining if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage comprises examining the data change map data structure.

12. The computer-implemented method of claim 11,

wherein the data change map data structure comprises a plurality of entries, wherein the plurality of entries track locations of changes to the volume that have not yet been replicated to a secondary storage;

wherein maintaining the data change map data structure in memory comprises merging entries in the data change map data structure if a number of entries in the data change map data structure reaches a limit.

13. The computer-implemented method of claim 8,

wherein periodically replicating changes to the volume to the secondary storage using the replication log further comprises:

the computer system reading a first record from the replication log, wherein the first record refers to a first area of the volume;

the computer system determining if at least a portion of the first area of the volume has been overwritten;

if at least a portion of the first area of the volume has not been overwritten, the computer system reading first data from the portion of the first area of the volume that has not been overwritten and providing the first data to the secondary storage;

if at least a portion of the first area of the volume has been overwritten, the computer system reading second data corresponding to the overwritten portion of the first area of the volume from the replication log and providing the second data to the secondary storage.

14. The computer-implemented method of claim 13,

wherein determining if at least a portion of the first area of the volume has been overwritten further comprises:

utilizing information in one or more data structures to determine if at least a portion of the first area of the volume has possibly been overwritten and to determine one or more portions of the replication log that comprise the second data.

15. A system for replicating data, the system comprising:

one or more processors;

a non-transitory computer-readable storage medium comprising program instructions executable by the one or more processors, wherein the program instructions are executable to;

receive a write request to write data to a location in a volume;

store metadata associated with the write request;

maintain information indicating locations of earlier write requests to the volume that have not yet been replicated to a secondary storage;

determine if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage;

store the data in a replication log in response to determining that the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume;

wherein the data is not stored in the replication log in response to determining that the write request does not overlap with one or more locations indicated by the information indicating locations of earlier write requests to the volume;

write the data to the location in the volume in response to the write request; and

periodically replicate changes to the volume to the secondary storage using the replication log.

16. The system of claim 15, wherein the program instructions executable to determine if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage are further executable to:

determine if the write request is a first write request that overlaps on a block range of one or more earlier write requests to the volume and that the one or more earlier write requests have not yet been replicated to the secondary storage;

wherein in response to determining that the write request is the first write request that overlaps on the block range, the program instructions are further executable to store data from the volume corresponding to the block range to the replication log.

17. The system of claim 16, wherein the program instructions are further executable to:

store linking information, wherein the linking information links an entry in the replication log comprising the data from the volume corresponding to the block range and one or more entries in the replication log corresponding to one or more overlapping earlier write requests.

18. The system of claim 15, wherein the program instructions are further executable to:

maintain a data change map data structure in memory, wherein the information indicating locations of earlier write requests to the volume that have not yet been replicated to a secondary storage comprises the data change map data structure;

wherein the program instructions are executable to examine the data change map data structure to determine if the write request overlaps with one or more locations indicated by the information indicating locations of earlier write requests to the volume that have not yet been replicated to the secondary storage.

19. The system of claim 15,

wherein, in periodically replicating changes to the volume to the secondary storage using the replication log, the program instructions are executable to:

read a first record from the replication log, wherein the first record refers to a first area of the volume;

determine if at least a portion of the first area of the volume has been overwritten;

if at least a portion of the first area of the volume has not been overwritten, then read first data from the portion of the first area of the volume that has not been overwritten and provide the first data to the secondary storage;

if at least a portion of the first area of the volume has been overwritten, then read second data corresponding to the overwritten portion of the first area of the volume from the replication log and provide the second data to the secondary storage.

20. The system of claim 19,

wherein, in determining if at least a portion of the first area of the volume has been overwritten, the program instructions are executable to:

utilize information in one or more data structures to determine if at least a portion of the first area of the volume has possibly been overwritten and to determine one or more portions of the replication log that comprise the second data.