On demand consistency checkpoints for temporal volumes within consistency interval marker based replication

• US 7,885,923 B1
• Filed: 06/13/2007
• Issued: 02/08/2011
• Est. Priority Date: 06/30/2006
• Status: Expired due to Fees

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (Gen Digital Inc.)
• Inventors: Karr, Ronald S., Tawri, Deepak
• Primary Examiner(s): Vy; Hung T
• Application Number:
  US11/762,608
• Time in Patent Office: 1,336 Days
• Field of Search: 707/609, 707/610, 707/611, 707/613, 707/615, 707/638, 707/639, 707/999.2, 711/114, 711/162, 714/19
• US Class Current: 707/610
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes

Abstract

In distributed storage environments, nodes may send replicated writes for temporal volumes and may be configured to request on-demand consistency intervals in order to allow multiple nodes to transition between timestamps for temporal data in a uniform and coordinated manner. In response to a request from a node for a consistency interval transition, a consistency interval coordinator may end a current consistency interval and start another and may provide coordination to various nodes to enable the nodes to transition from one timestamp to another, and thus from one temporal version to another, while prevent data corruption or inconsistency between the temporal versions. Additionally, periodic consistency intervals may be utilized to provide consistent and coordinated transitions between temporal timestamps. In some embodiments, replicated writes including temporal timestamp information may be stored persistent spillover logs allowing offline updating and versioning of data to temporal volumes.

Claims

1. A method, comprising:

a plurality of nodes in a distributed computing environment sending write requests to a primary temporal volume and replicated write requests for the temporal volume to a replication target device, wherein the temporal volume is configured to store temporal data, wherein each replicated write request is a copy of a corresponding one of the write requests, wherein each of the replicated write requests is associated with an indication of a timestamp for the temporal data on the temporal volume, and wherein the replication target device is configured to generate a checkpoint of the temporal volume corresponding to the timestamp;

initiating a consistency interval transition, wherein the consistency interval transition defines a point in time at which data on the temporal volume is in a consistent state, and wherein the consistency interval transition corresponds to the timestamp;

in response to the initiation of the consistency interval transition, each of the plurality of nodes;

temporarily suspending sending replicated write requests to the replication target device; and

subsequent to said temporarily suspending, resuming sending replicated write requests to the replication target device, wherein an indication of a second timestamp is associated with each replicated write request sent after said resuming.

2. The method as recited in claim 1, further comprising:

an application in the distributed computing environment sending a new timestamp message to each of the plurality of nodes, wherein the new timestamp message indicates a new timestamp;

in response to receiving the new timestamp message from the application, each of the plurality of nodes sending a checkpoint request message to a consistency interval coordinator; and

in response to receiving the checkpoint request message from each of the plurality of nodes, the consistency interval coordinator performing said initiating the consistency interval transition.

3. The method as recited in claim 2, further comprising the consistency interval coordinator, in further response to receiving the checkpoint request message from each of the plurality of nodes, sending an acknowledgment message to each of the plurality of nodes.

4. The method as recited in claim 3, further comprising each of the plurality of nodes performing said resuming sending replicated writes in response to receiving the acknowledgement message from the consistency interval coordinator.

5. The method as recited in claim 1, wherein said sending replicated write requests comprises:

storing each of the replicated write requests in one or more logs;

reading each of the replicated write requests from the one or more logs; and

storing data corresponding to each of the replicated write requests to the temporal volume.

6. The method as recited in claim 5, further comprising:

wherein said storing each of the replicated write requests in one or more logs further comprises storing timestamp information associating each stored replicated write request with a respective timestamp; and

creating, on the temporal volume, at least one temporal version of data for the stored replicated write requests according to the stored timestamp information.

7. The method as recited in claim 1, further comprising:

storing data corresponding to each of the replicated write requests as temporal data on the temporal volume; and

associating the data corresponding to each of the replicated write requests with a corresponding timestamp associated with each respective replicated write request.

8. A distributed storage system, comprising:

a plurality of nodes;

a replication target device; and

a consistency interval coordinator;

wherein each of the plurality of nodes is configured to send write requests to a primary temporal volume and replicated write requests for the temporal volume to a replication target device, wherein the temporal volume is configured to store temporal data, wherein each replicated write request is a copy of a corresponding one of the write requests, wherein each of the replicated write requests is associated with an indication of a timestamp for the temporal data on the temporal volume, and wherein the replication target device is configured to generate a checkpoint of the temporal volume corresponding to the timestamp;

wherein the consistency interval coordinator is configured to initiate a consistency interval transition, wherein the consistency interval transition defines a point in time at which data on the temporal volume is in a consistent state and wherein the consistency interval transition corresponds to the timestamp;

wherein in response to the initiation of the consistency interval transition, each of the plurality of nodes is further configured to;

temporarily suspend sending replicated write requests to the replication target device; and

subsequent to said temporarily suspending, resume sending replicated write requests to the replication target device, wherein an indication of a second timestamp is associated with each replicated write request sent after said resuming.

9. The system of claim 8, further comprising:

an application executing on at least one of the plurality of nodes, wherein the application is configured to send a new timestamp message to each of the plurality of nodes, wherein the new timestamp message indicates a new timestamp;

wherein in response to receiving the new timestamp message from the application, each of the plurality of nodes is further configured to send a checkpoint request message to a consistency interval coordinator; and

wherein in response to receiving the checkpoint request message from each of the plurality of nodes, the consistency interval coordinator is configured to perform said initiating the consistency interval transition.

10. The system of claim 9, wherein the consistency interval coordinator, in further response to receiving the checkpoint request message from each of the plurality of nodes, is further configured to send an acknowledgment message to each of the plurality of nodes.

11. The system of claim 10, wherein each of the plurality of nodes is further configured to perform said resuming sending replicated writes in response to receiving the acknowledgement message from the consistency interval coordinator.

12. The system of claim 8, wherein each of the plurality of nodes is configured to store each of the replicated write requests in one or more logs;

and

wherein the replication target device is configured to;

read each of the replicated write requests from the one or more logs; and

store data corresponding to each of the replicated write requests to the temporal volume.

13. The system of claim 12, wherein each of the plurality of nodes is further configured to:

store timestamp information associating each stored replicated write request with a respective timestamp; and

wherein the replication target device is configured to create, on the temporal volume, at least one temporal version of data for the stored replicated write requests according to the stored timestamp information.

14. The system of claim 8, wherein the replicated target device is further configured to:

store data corresponding to teach of the replicated write requests as temporal data on the temporal volume; and

associate the data corresponding to each of the replicated write requests with a corresponding timestamp associated with each respective replicated write request.

15. A device, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises program instructions executable by the processor to;

send write requests to a primary temporal volume and replicated write requests for the temporal volume to a replication target device, wherein the temporal volume is configured to store temporal data, wherein each replicated write request is a copy of a corresponding one of the write requests, wherein each of the replicated write requests is associated with an indication of a timestamp for the temporal data on the temporal volume, and wherein the replication target device is configured to generate a checkpoint of the temporal volume corresponding to the timestamp;

wherein in response to an initiation of a consistency interval transition, wherein the consistency interval transition defines a point in time at which data on the temporal volume is in a consistent state, and wherein the consistency interval transition corresponds to the timestamp, each of the plurality of nodes is further configured to;

temporarily suspend sending replicated write requests to the replication target device; and

subsequent to said temporarily suspending, resume sending replicated write requests to the replication target device, wherein an indication of a second timestamp is associated with each replicated write request sent after said resuming.

16. The device of claim 15, wherein the program instruction are further executable by the processor to send a checkpoint request message to a consistency interval coordinator in response to receiving a new timestamp message indicating a new timestamp.

17. The device of claim 16, wherein the program instructions are further executable by the processor to resume sending replicated writes in response to receiving an acknowledgement message from the consistency interval coordinator.

18. The device of claim 15, wherein the program instructions are further executable by the processor to store each of the replicated write requests in at least one persistent spillover logs for subsequent updating of at least one temporal version of data on the temporal volume.

19. The device of claim 18, wherein the program instructions are further executable by the processor to store timestamp information associating each stored replicated write request with a respective timestamp in the least one persistent spillover logs, wherein the stored timestamp information is configured for use in creating at least one temporal version of data for the stored replicated write requests.