System and method for creating an application-consistent remote copy of data using remote mirroring

• US 20070022144A1
• Filed: 07/21/2005
• Published: 01/25/2007
• Est. Priority Date: 07/21/2005
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Ying
• Granted Patent: US 7,464,126 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

An application consistent data protection system provides application-assist and replication-technology neutral mirroring that ensures that a remote data copy is application-consistent. The system comprises a coordination protocol to coordinate application hosts across heterogeneous hosts and heterogeneous storage devices. The system utilizes a disk layout and data record format that enables use of an underlying replication ability of a storage device, minimizing development cost and utilizing customer investment. The system comprises on-demand consistency point initiation to minimize performance impact and maximize system resource usage. The system can be applied to both synchronous and asynchronous mirroring and can be incorporated into any virtualization device.

Claims

1. A method of creating an application-consistent remote copy of data using remote mirroring, comprising:

intercepting a write request from at least one of a plurality of application hosts;

appending the intercepted write request as a write record in a write record format to a local log in a local replication volume;

instructing the application hosts to prepare a consistency point;

generating a consistency point record to identify a set of write records comprising a consistency point data set;

replicating the write records and the consistency point records to a remote log in a remote replication volume;

scanning the remote log for the consistency point record;

verifying the validity of the write records in the remote log, to generate a validated consistency point update transaction; and

writing the validated consistency point update transaction to a remote storage device to generate an application-consistent remote copy of the consistency point data set.

2. The method of claim 1, wherein the write record format comprises any one or more of:

a head hash signature;

a tail hash signature;

a request ID;

a record ID;

a record type;

an intercept agent ID;

a timestamp;

a sequence number;

a hash bock starting offset, S;

a disk block address; and

data from the write request.

3. The method of claim 2, wherein the record ID is used by a remote agent to identify the write records and the consistency point records.

4. The method of claim 2, wherein the record type is used by a remote agent to identify the write records and the consistency point records.

5. The method of claim 2, wherein the sequence number increases sequentially.

6. The method of claim 2, wherein the hash bock starting offset is dynamically and randomly generated.

7. The method of claim 2, wherein the head hash signature and the tail hash signature are calculated as hash values of the timestamp, the sequence number, the intercept agent ID, and a predetermined number, N, bytes of the data starting at the hash block starting offset, S.

8. The method of claim 2, further comprising combining an append-only circular log disk layout and the data record format with the content-dependent head hash signature and the tail hash signature hash signature, and uniquely verifiable fields, to allow a remote data processing agent to extract data and to generate a remote copy in a consistency-preserving fashion.

9. The method of claim 8, wherein combining the append-only circular log disk layout comprises appending records at the end of circular log, immediately following a previously appended record.

10. The method of claim 8, wherein the append-only circular log comprises one or more consistency point data sets.

11. The method of claim 10, further comprising deleting each consistency point data set after being written from the remote log in the remote replication volume to the remote storage.

12. The method of claim 1, wherein instructing the application hosts to prepare the consistency point comprises the application hosts quiesce application updates.

13. The method of claim 1, wherein instructing the application hosts to prepare the consistency point comprises the application hosts completing transient transactions.

14. The method of claim 1, wherein instructing the application hosts to prepare the consistency point comprises the application hosts flushing data in write buffers to the local storage device.

15. The method of claim 1, wherein the consistency point record comprises any one or more of:

a consistency point head hash signature that is calculated as a for write record;

a consistency point tail hash signature that is calculated as a for write record;

a record type;

an intercept agent ID;

a consistency point timestamp;

a consistency point sequence number;

a hash block starting offset, S;

a last timestamp of the consistency point data set; and

a last sequence number of the consistency point data set.

16. The method of claim 1, wherein scanning the remote log comprises looking for holes.

17. The method of claim 16, wherein scanning the remote log comprises pausing until the holes are filled.

18. The method of claim 1, wherein after writing the validated consistency point update transaction deleting the write records in the consistency point data set and the consistency point update transaction.

19. A computer program product having a plurality of executable instruction codes that are stored on a computer-readable medium, for creating an application-consistent remote copy of data using remote mirroring, comprising:

a first set of instruction codes for intercepting a write request from at least one of a plurality of application hosts;

a second set of instruction codes for appending the intercepted write request as a write record in a write record format to a local log in a local replication volume;

a third set of instruction codes for instructing the application hosts to prepare a consistency point;

a fourth set of instruction codes for generating a consistency point record to identify a set of write records comprising a consistency point data set;

a fifth set of instruction codes for replicating the write records and the consistency point records to a remote log in a remote replication volume;

a sixth set of instruction codes for scanning the remote log for the consistency point record;

a seventh set of instruction codes for verifying the validity of the write records in the remote log, to generate a validated consistency point update transaction; and

an eight set of instruction codes for writing the validated consistency point update transaction to a remote storage device to generate an application-consistent remote copy of the consistency point data set.

20. A system for creating an application-consistent remote copy of data using remote mirroring, comprising:

an intercept agent for intercepting a write request from at least one of a plurality of application hosts;

the intercept agent further appending the intercepted write request as a write record in a write record format to a local log in a local replication volume;

a replication coordinator for instructing the application hosts to prepare a consistency point;

the replication coordinator generating a consistency point record to identify a set of write records comprising a consistency point data set;

replicating the write records and the consistency point records to a remote log in a remote replication volume;

a remote agent for scanning the remote log for the consistency point record;

the remote agent verifying the validity of the write records in the remote log, to generate a validated consistency point update transaction; and

the remote agent further writing the validated consistency point update transaction to a remote storage device to generate an application-consistent remote copy of the consistency point data set.