OVERLAPPING WRITE DETECTION AND PROCESSING FOR SYNC REPLICATION

• US 20160062685A1
• Filed: 08/29/2014
• Published: 03/03/2016
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Mu, Yuedong, Coatney, Susan M., Sundararajan, Manoj V., Ngan, Ching-Yuk Paul
• Granted Patent: US 9,645,753 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A primary write request that is to modify a primary portion of primary data stored in a primary storage node is received. The primary write request is to be replicated to create a current secondary write request. The current secondary write request is to modify a current secondary portion of secondary data that is stored in a secondary storage node. A current data range of the current secondary portion is determined. A determination is made of whether a previous secondary write request is in process of modifying a previous data range that at least partially overlaps with a current data range of the current secondary portion. Execution of the primary write request is suspended, until the previous secondary write request has completed updating the secondary storage node.

Claims

1. A method comprising:

receiving, at a primary storage node, a primary write request that is to modify a primary portion of primary data stored in the primary storage node, wherein the primary write request is to be replicated to create a current secondary write request, wherein the current secondary write request is to modify a current secondary portion of secondary data that is stored in a secondary storage node, wherein the secondary data comprises a replication of the primary data;

determining a current data range of the current secondary portion;

determining whether a previous secondary write request, that has not yet completed updating the secondary storage node, is in process of modifying a previous data range that at least partially overlaps with the current data range; and

suspending execution of the primary write request, until the previous secondary write request has completed updating the secondary storage node.

2. The method of claim 1, wherein a plurality of other primary write requests at the primary storage node are suspended from execution.

3. The method of claim 2, further comprising:

performing the following, after the previous secondary write request has completed execution,

determining whether at least one of the plurality of other primary write requests is suspended and is configured to modify a data range that overlaps with the previous data range;

selecting, in response to determining that the at least one of the plurality of other primary write requests is suspended and is configured to modify a data range that overlaps with the previous data range, a selected primary write request from among the primary write request and the at least one of the plurality of other primary write requests that has been suspended from execution a longest time; and

resuming execution of the selected primary write request, wherein resuming execution of the selected primary write request comprises replicating the selected primary write request, external to the filesystem, to create the current secondary write request for transmission to the secondary storage node in response to the selected primary write request being the primary write request.

4. The method of claim 3, wherein the replicating occurs after translating a protocol of the primary write access request into a unified protocol.

5. The method of claim 2, wherein performing the following, after the previous secondary write request has completed execution is in response to receiving a sync completed message from the secondary storage node to notify the primary storage node that the secondary portion has been updated.

6. The method of claim 2, further comprising storing, while suspended from execution, the plurality of other primary write requests and the primary write request into a shared suspended queue for updates to different portions of the secondary data.

7. The method of claim 2,

wherein each portion of the secondary data is assigned a suspended queue of a plurality of suspended queues,

wherein the method further comprises storing, while suspended from execution, each of the plurality of other primary write requests and the primary write request into one of the plurality of suspended queues assigned to a portion of the secondary data that is to be updated.

8. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions comprising program instructions to:

receive, at a primary storage node, a primary write request that is to modify a primary portion of primary data stored in the primary storage node, wherein the primary write request is to be replicated, external to a file system, to create a current secondary write request wherein the current secondary write request is to modify a current secondary portion of secondary data that is stored in a secondary storage node, wherein the secondary data comprises a replication of the primary data;

determine a current data range of the current secondary portion;

determine whether a previous secondary write request, that has not yet completed an update of the secondary storage node, is in process to modify a previous data range that at least partially overlaps with the current data range; and

suspend execution of the primary write request, until the previous secondary write request has completed the update of the secondary storage node.

9. The computer program product of claim 8, wherein a plurality of other primary write requests received by the primary storage node are suspended from execution.

10. The computer program product of claim 9, wherein the program instructions are to

perform the following, after the previous secondary write request has completed execution,

determine whether at least one of the plurality of other primary write requests is suspended and is configured to modify a data range that overlaps with the previous data range;

select, in response to a determination that the at least one of the plurality of other primary write requests is suspended and is configured to modify a data range that overlaps with the previous data range, a selected primary write request from among the primary write request and the at least one of the plurality of other primary write requests that has been suspended from execution a longest time; and

resume execution of the selected primary write request, wherein the resume of execution of the selected primary write request comprises a replication of the selected primary write request to create the current secondary write request for transmission to the secondary storage node in response to the selected primary write request being the primary write request.

11. The computer program product of claim 9, wherein the program instructions are to perform the following, after the previous secondary write request has completed execution in response to receipt of a sync completed message from the secondary storage node to notify the primary storage node that the secondary portion has been updated.

12. The computer program product of claim 9, wherein the program instructions are to store, while suspended from execution, the plurality of other primary write requests and the primary write request into a shared suspended queue for updates to different portions of the secondary data.

13. The computer program product of claim 9,

wherein each portion of the secondary data is assigned a suspended queue of a plurality of suspended queues,

wherein the program instructions are to store, while suspended from execution, each of the plurality of other primary write requests and the primary write request into one of the plurality of suspended queues assigned to a portion of the secondary data that is to be updated.

14. The computer program product of claim 8, wherein the primary storage node is part of a primary storage cluster, and wherein the secondary storage node is part of a secondary storage cluster.

15. A system comprising:

a primary storage node comprising,

a primary processor;

a primary storage configured to store primary data; and

one or more primary computer readable storage media having primary program instructions embodied therewith, the primary program instructions executable by the primary processor to cause the primary storage node to,

receive a primary write request to modify a primary portion of primary data that is stored in a primary storage node, wherein the primary write request is to be replicated external to a filesystem to create a current secondary write request, wherein the current secondary write request is to modify a current secondary portion of secondary data that is stored in a secondary storage node, wherein the secondary data comprises a replication of the primary data; and

determine a current data range of the secondary portion based on the secondary write request;

determine whether a previous secondary write request that has not yet completed an update of the secondary storage node is in process of an update to a previous data range that at least partially overlaps with the current data range;

suspend execution of the primary write request, until at least the previous secondary write request has completed the update to the secondary storage node;

replicate the primary write request to create the secondary write request for transmission; and

transmit the secondary write request to the secondary storage node for execution;

the secondary storage node comprising,

a secondary processor;

a secondary storage configured to store secondary data that comprises a replication of the primary data; and

one or more secondary computer readable storage media having secondary program instructions embodied therewith, the secondary program instructions executable by the secondary processor to cause the secondary storage node to,

receive the secondary write request; and

update the secondary portion of the secondary data, in response to receipt of the secondary write request.

16. The system of claim 15, wherein a plurality of other primary write requests configured to be received by the primary storage node are to be suspended from execution upon receipt.

17. The system of claim 16, wherein the primary program instructions executable by the primary processor further comprises primary program instructions executable by the primary processor to cause the primary storage node to,

perform the following, after the previous secondary write request has completed an update of the secondary storage node,

determine whether at least one of the plurality of other primary write requests is suspended and is configured to modify a data range that overlaps with the previous data range;

select, in response to a determination that the at least one of the plurality of other primary write requests is suspended and is configured to modify a data range that overlaps with the previous data range, a selected primary write request from among the primary write request and the at least one of the plurality of other primary write requests that has been suspended from execution a longest time; and

resume execution of the selected primary write request.

18. The system of claim 16, wherein the primary program instructions executable by the primary processor to cause the primary storage node to perform the following, after the previous secondary write request has completed the update of the secondary data, is in response to a sync completed message from the secondary storage node to notify the primary storage node that the secondary portion has been updated.

19. The system of claim 16, wherein the primary program instructions executable by the primary processor further comprises primary program instructions executable by the primary processor to cause the primary storage node to store, while suspended from execution, the plurality of other primary write requests and the primary write request into a shared suspended queue for updates to different portions of the secondary data.

20. The system of claim 15,

wherein the primary program instructions executable by the primary processor comprise primary program instructions executable by the primary processor to cause the primary storage node to translate a protocol of the primary write request into a unified protocol, and

wherein the primary program instructions executable by the primary processor to cause the primary storage node to replicate the primary write request comprises primary program instructions executable by the primary processor to cause the primary storage node to replicate, after translation of the protocol of the primary write request, the primary write request.