SYNCHRONIZATION OF SNAPSHOTS IN A DISTRIBUTED STORAGE SYSTEM

• US 20160205182A1
• Filed: 01/12/2015
• Published: 07/14/2016
• Est. Priority Date: 01/12/2015
• Status: Active Grant

• Current Assignee: Mellanox Technologies Limited (NVIDIA Corporation)
• Original Assignee: Stratoscale Ltd.
• Inventors: Feldmann, Yael, Lazar, Aharon
• Granted Patent: US 9,912,748 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method includes storing data generated in a source node by sending write requests to multiple destination nodes. The destination nodes are requested to create snapshots of the data. The write requests are marked at the source node with marks that indicate to each destination node which of the write requests are pre-snapshot write requests that were issued before a snapshot request for a snapshot that the destination node is currently storing, and which of the write requests are post-snapshot write requests that were issued after the snapshot request for the snapshot that the destination node is currently storing. The snapshots are synchronized with one another at the destination nodes based on the marks.

Claims

1. A method, comprising:

storing data, which belongs to a logical volume and is generated in a source node, by sending write requests associated with the logical volume to multiple destination nodes;

requesting the destination nodes to create snapshots of the data;

at the source node, marking the write requests with marks that indicate to each destination node which of the write requests are pre-snapshot write requests that were issued before a snapshot request for a snapshot that the destination node is currently storing, and which of the write requests are post-snapshot write requests that were issued after the snapshot request for the snapshot that the destination node is currently storing; and

at the destination nodes, synchronizing the snapshots with one another based on the marks, by;

individually in each of the destination nodes, distinguishing between the pre-snapshot write requests and the post-snapshot write requests based on the marks, irrespective of differences in order-of-arrival of the write requests and the snapshot request among the destination nodes; and

individually in each of the destination nodes, serving only the pre-snapshot write requests.

2. The method according to claim 1, wherein sending the write requests comprises sending to the multiple destination nodes multiple respective replicas of same data.

4. The method according to claim 1, and comprising, in each destination node, queuing each post-snapshot write request until becoming a pre-snapshot write request due to arrival of a subsequent snapshot request.

5. The method according to claim 1, wherein synchronizing the snapshots comprises, in each destination node, queuing the write requests and serving only the queued write requests that become pre-snapshot write requests.

6. The method according to claim 1, wherein marking the write requests comprises tagging each write request with a respective counter value indicating the snapshot that was most recently requested by the source node.

7. The method according to claim 6, wherein synchronizing the snapshots comprises, in each destination node, distinguishing between the pre-snapshot write requests and the post-snapshot write requests by comparing the respective counter value of each write request with a reference counter value of the snapshot that the destination node is currently storing.

3. (canceled)

8. A system, comprising at least a source node and multiple destination nodes,

wherein the source node is configured to store data, which belongs to a logical volume, by sending write requests associated with the logical volume to the destination nodes, and to mark the write requests with marks that indicate to each destination node which of the write requests are pre-snapshot write requests that were issued before a snapshot request for a snapshot that the destination node is currently storing, and which of the write requests are post-snapshot write requests that were issued after the snapshot request for the snapshot that the destination node is currently storing,

and wherein the destination nodes are configured to synchronize snapshots with one another based on the marks, by:

individually in each of the destination nodes, distinguishing between the pre-snapshot write requests and the post-snapshot write requests based on the marks, irrespective of differences in order-of-arrival of the write requests and the snapshot request among the destination nodes; and

individually in each of the destination nodes, serving only the pre-snapshot write requests.

9. The system according to claim 8, wherein the source node is configured to send to the multiple destination nodes multiple respective replicas of same data.

11. The system according to claim 8, wherein each destination node is configured to queue each post-snapshot write request until becoming a pre-snapshot write request due to arrival of a subsequent snapshot request.

12. The system according to claim 8, wherein each destination node is configured to queue the write requests and to serve only the queued write requests that become pre-snapshot write requests.

13. The system according to claim 8, wherein the source node is configured to mark each write request with a respective counter value indicating the snapshot that was most recently requested by the source node.

14. The system according to claim 13, wherein each destination node is configured to distinguish between the pre-snapshot write requests and the post-snapshot write requests by comparing the respective counter value of each write request with a reference counter value of the snapshot that the destination node is currently storing.

10. (canceled)

15. A computer software product, the product comprising a tangible non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by processors of a source node and multiple destination nodes that are interconnected by a communication network, cause the processors to store data, which belongs to a logical volume and is generated in the source node, by sending write requests associated with the logical volume to the multiple destination nodes, to request the destination nodes to create snapshots of the data, to mark the write requests at the source node with marks that indicate to each destination node which of the write requests are pre-snapshot write requests that were issued before a snapshot request for a snapshot that the destination node is currently storing, and which of the write requests are post-snapshot write requests that were issued after the snapshot request for the snapshot that the destination node is currently storing, and to synchronize the snapshots with one another at the destination nodes based on the marks, by:

individually in each of the destination nodes, distinguishing between the pre-snapshot write requests and the post-snapshot write requests based on the marks, irrespective of differences in order-of-arrival of the write requests and the snapshot request among the destination nodes; and

individually in each of the destination nodes, serving only the pre-snapshot write requests.