Write ordering style asynchronous replication utilizing a loosely-accurate global clock

• US 7,925,629 B2
• Filed: 03/28/2007
• Issued: 04/12/2011
• Est. Priority Date: 03/28/2007
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Webman, Erez, Koren, Dvir, Novick, Yoram
• Primary Examiner(s): Wu; Yicun
• Assistant Examiner(s): Shanmugasundaram; Kannan
• Application Number:
  US11/692,668
• Publication Number:
  US 20080243951A1
• Time in Patent Office: 1,476 Days
• Field of Search: 707/204
• US Class Current: 707/648
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2074 Asynchronous techniques

Abstract

A write ordering style asynchronous replication service utilizes a loosely-accurate global clock in a data backup and recovery storage environment. The storage environment includes a set of source storage systems illustratively embodied as source replication nodes that cooperate to maintain a consistency group that may span multiple geographical sites. The storage environment also includes one or more target storage systems illustratively embodied as target replication nodes configured to service the consistency group. The write ordering style service utilizes the loosely-accurate global clock to provide consistent replication of a storage space of the consistency group.

Claims

1. A method for providing a write ordering style asynchronous replication service, comprising:

embodying a set of source storage systems as source replication nodes that cooperate to maintain a consistency group that spans multiple geographical sites, each source storage system comprising a first processor configured to execute a first storage operating system of at least one source storage system;

embodying a target storage system as a target replication node configured to service the consistency group, the target storage system comprising a second processor configured to execute a second storage operating system of the target storage system;

intercepting a storage request at a write interception component of a particular source replication node;

recording the storage request at the particular source replication node;

generating metadata for the storage request at the particular source replication node;

assigning a value to the storage request according to a clock of the particular source replication node;

propagating the storage request from the particular source replication node to the target replication node and storing the propagated storage request as a log entry of a plurality of log entries of the target replication node;

establishing a consistency timeline value at the target replication node using a first predetermined parameter and a second predetermined parameter, wherein a first log entry of the plurality of log entries has an assigned value up to the consistency timeline value and a second log entry of the plurality of log entries has an assigned value greater than the consistency timeline value; and

applying the first log entry to a storage space of the target replication node in response to the first log entry having the assigned value up to the consistency timeline value, wherein the second log entry having the assigned value greater than the consistency timeline value is not applied to the storage space.

2. The method of claim 1 wherein propagating further comprises:

propagating the storage request as the log entry of a target replication node log from the particular source replication node to the target replication node for storage on the storage space.

3. The method of claim 1 wherein propagating further comprises:

propagating each log entry to a target replication node log of the target replication node; and

storing each log entry in the target replication node log.

4. The method of claim 1 further comprising:

advancing the consistency timeline value by a consistency engine of the target replication node.

5. The method of claim 1 wherein the first and the second predetermined parameters comprise T-info and Minimum T-info parameters.

6. The method of claim 1 wherein applying the first log entry comprises applying the first log entry by an apply engine of the target replication node.

7. The method of claim 1 further comprising:

applying the first log entry up to the consistency timeline to create a crash image on the storage space of the target replication node.

8. The method of claim 1 further comprising:

basing a global clock of one of the source replication nodes on a timestamp and an associated error boundary.

17. The system of claim 8 wherein the clock is based, at least in part, on a timestamp and an associated error boundary.

9. A system configured to provide a write ordering style asynchronous replication service, comprising:

a set of source storage systems configured as source replication nodes that cooperate to maintain a consistency group that spans multiple geographical sites, each source storage system comprising a first processor configured to execute a first storage operating system of at least one source storage system;

a target storage system configured as a target replication node configured to service the consistency group, the target storage system comprising a second processor configured to execute a second storage operating system of the target storage system;

a write interception component of one or more source replication nodes configured to intercept a storage requests, and further configured to record the requests, and further configured to generate metadata for the requests;

the source replication nodes configured to assign a values to the storage requests, according to a clock of the source replication node that intercepted the storage request;

one or more entries of a log of target replication node configured to receive the storage request from the source replication node that intercepted the storage request;

a consistency timeline value at the target replication node using a first predetermined parameter and a second predetermined parameter, wherein a first log entry has an assigned value up to the consistency timeline value and a second log entry has an assigned value greater than the consistency timeline value; and

the target replication node further configured to apply the first log entry to a storage space of the target replication node in response to the first log entry having the assigned value up to the consistency timeline value, wherein the second log entry having the assigned value greater than the consistency timeline value is not applied to the storage space.

10. The system of claim 9 wherein the storage request comprises a write operation.

11. The system of claim 9 wherein the clock is comprises a global clock.

12. The system of claim 9 wherein the target replication node further comprises a consistency engine configured to establish, maintain and advance the consistency timeline value.

13. The system of claim 9 wherein the first and the second predetermined parameters comprise T-info and Minimum T-info parameters.

14. The system of claim 13 wherein the T-info is a guarantee that all storage requests have been propagated by each source replication node and wherein the Minimum T-info comprises the T-info with a lowest value.

15. The system of claim 9 wherein the target replication node further comprises an apply engine configured to apply any log entry to the storage space up to the consistency timeline value to create a crash image on the target storage space.

16. The system of claim 15 wherein the crash image comprises a storage image of the storage space of the target replication node that is available at restart from an interruption.

18. A computer--readable storage medium containing executable program instructions executed by a processor, comprising:

program instructions that embody a set of source storage systems as source replication nodes that cooperate to maintain a consistency group that spans multiple geographical sites, each source storage system comprising a first processor configured to execute a first storage operating system of at least one source storage system;

program instructions that embody a target storage system as a target replication nodes configured to service the consistency group, the target storage system comprising a second processor configured to execute a second storage operating system of the target storage system;

program instructions that intercept a storage request at a write interception component of a particular source replication node;

program instructions that record the storage request at the particular source replication node;

program instructions that generate metadata for the storage request at the particular source replication node;

program instructions that assign a value to the storage requests according to a clock of the particular source replication node;

program instructions that propagate the storage request from the particular source replication node to the target replication node and program instructions that store the storage request as a log entry of a plurality of log entries of the target replication node;

program instructions that establish a consistency timeline value at the target replication node using a first predetermined parameter and a second predetermined parameter, wherein a first log entry of the plurality of log entries has an assigned value up to the consistency timeline value and a second log entry of the plurality of log entries has an assigned value greater than the consistency timeline value; and

program instructions that apply the first log entry to a storage space of the target replication node in response to the first log entry having the assigned value up to the consistency timeline value, wherein the second log entry having the assigned value greater than the consistency timeline value are not applied to the storage space.

19. The computer--readable storage medium of claim 18 further comprising:

program instructions that base a global clock of one of the source replication nodes on a timestamp and an associated error boundary.

20. A system for providing a write ordering asynchronous replication service, comprising:

a set of source storage systems configured as source replication nodes (SRNs), the SRNs configured to cooperate to maintain a consistency group, each source storage system comprising a first processor configured to execute a first storage operating system of at least one source storage system;

a target storage systems configured to be embodied as a target replication nodes (TRN), the TRN configured to service the consistency group, each target storage system comprising a second processor configured to execute a second storage operating system of at least one target storage system;

a write interception component of a particular SRN configured to intercept a storage request directed to the consistency group and further configured to record the storage request, and further configured to generate and assign a timestamp value associated with a clock of the particular SRN to the storage request, wherein the timestamp value has a maximum error boundary;

the TRN further configured to receive the storage request with the timestamp and the, maximum error boundary propagated from the particular SRN;

a consistency engine of the TRN configured to establish a current consistency timeline value using a first predetermined parameter and a second predetermined parameter; and

an apply engine of the TRN configured to apply storage request received from the particular SRN to a target storage space of the consistency group when the timestamp and the maximum error boundary of the storage request is earlier than the current consistency timeline value.

21. The system of claim 20 further comprising:

the consistency engine further configured to advance the consistency timeline value.

22. The system of claim 20 wherein the first and the second predetermined parameters are comprise T-info and Minimum T-info parameters.

23. The system of claim 20 further comprising a global clock of a master SRN configured to provide consistent replication of the target storage space, wherein the global clock has no error boundary.

24. The system of claim 20 further comprising:

the TRN further configured to receive a plurality of storage requests in a first order; and

the TRNs further configured to reorder the plurality of storage requests to be written in a second order.

25. The system of claim 20 further comprising:

one or more of the SRNs configured with a shared-everything relationship; and

the one or more of the SRNs configured with the shared-everything relationship configured to be re-routed to create a shared-nothing relationship instead of the shared-everything relationship.

26. The system of claim 20 further comprising:

one or more SRNs further configured to read data of an overlapping storage space address;

the one or more SRNs further configured to record a time that the data is read and when the data is finished being read; and

the consistency engine further configured to calculate another consistency timeline to a time later than when the data is finished being read.

27. The system of claim 20 further comprising:

introducing a quiescent period; and

one or more SRNs further configured to halt, sending at least one storage request to the TRN in response to the quiescent period.

28. A method for providing write ordering style asynchronous replication service, comprising:

embodying a set of source storage systems as source replication nodes (SRNs) that cooperate to maintain a consistency group, each source storage system comprising a first processor configured to execute a first storage operating system of at least one source storage system;

embodying a target storage system as a target replication node (TRN) configured to service the consistency group, the target storage system comprising a second processor configured to execute a second storage operating system of the target storage system;

intercepting a storage request directed to the consistency group at a write interception component of a particular SRN and recording the storage request;

generating by the write interception component a timestamp with a maximum error boundary of the timestamp associated with a clock of the particular SRN that intercepted the storage request;

propagating the storage request with the timestamp and the maximum error boundary from the particular SRN to the TRN;

establishing a current consistency timeline value at the TRN using a first parameter and a second parameter; and

applying the storage request propagated from the particular SRN to a target storage space of the TRN when the timestamp and the maximum error boundary of the storage request is earlier than the current consistency timeline value at the TRN.

29. The method of claim 28 further comprising:

basing a global clock of one of the SRNs on the timestamp and the associated error boundary.

30. The method of claim 28 further comprising:

a global clock of a master SRN configured to provide consistent replication of the target storage space of the consistency group, wherein the global clock has no error boundary.