Creating Snapshots Of A Storage Volume In A Distributed Storage System

• US 20190073372A1
• Filed: 09/05/2017
• Published: 03/07/2019
• Est. Priority Date: 09/05/2017
• Status: Active Grant

• Current Assignee: Robin Systems, Inc. (Rakuten Group, Inc.)
• Original Assignee: Robin Systems, Inc. (Rakuten Group, Inc.)
• Inventors: Venkatesan, Dhanashankar, Seetala, Partha Sarathi, Singh, Gurmeet
• Granted Patent: US 11,947,489 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A segment maps segments to a particular snapshot and metadata stored in the segment indicates storage volume addresses of data written to the segment. The snapshots may be represented by a storage manager in a hierarchy that identifies an ordering of snapshots and branches to clone snapshots.

Claims

1. A system comprising:

a plurality of storage nodes;

a network, the plurality of storage nodes in data communication with the plurality of storage nodes;

a storage manager in data communication with the network, the storage manager programmed to—

transmit, to one or more storage nodes of the plurality of storage nodes, a plurality of new snapshot instructions with respect to a storage volume, each new snapshot instruction being one of a new principal snapshot instruction and a new clone instruction and having a corresponding preceding snapshot node in a hierarchical representation of the storage volume;

for each new snapshot instruction of the plurality of new snapshot instructions, add a new snapshot node as a descendent to the corresponding preceding snapshot node of the each new snapshot instruction in the hierarchical representation, the new snapshot node having a clone classification if the each new snapshot instruction was a new clone instruction.

2. The system of claim 1, wherein the storage manager is further programmed to transmit the plurality of new snapshot instructions with respect to the storage volume such that the hierarchical representation includes a tree including:

(a) multiple snapshot nodes that are descendants of an initial snapshot node;

(b) at least one snapshot node that has the clone classification and one or more descendent nodes of the at least one snapshot node that has the clone classification; and

(c) at least one snapshot node that has a direct descendent that has the clone classification and a direct descendent that does not have the clone classification.

3. The system of claim 1, wherein the plurality of storage nodes are programmed to, in response to the new snapshot instruction:

allocate a new segment;

create an entry in a segment table mapping the new segment to a new snapshot;

write data from subsequent write requests for the storage volume referencing the new snapshot to the new segment.

4. The system of claim 3, wherein each storage node of the plurality of storage nodes is further programmed to:

receive a write instruction referencing a written snapshot, the write instruction including a write address;

identify a last-allocated segment corresponding to the written snapshot in the segment table;

write payload data from the write instruction to the last-allocated segment; and

write metadata to an index of the last-allocated segment corresponding to the written snapshot, the metadata including the write address.

5. The system of claim 4, wherein the storage manager is further programmed to:

issue one or more first new snapshot instructions of the plurality of snapshot instructions to a first storage node of the one or more storage nodes, the one or more first new snapshot instructions being principal snapshot instructions;

issue one or more second new snapshot instructions of the plurality of snapshot instructions to a second storage node of the one or more storage nodes, the one or more second new snapshot instructions being clone snapshot instructions.

6. The system of claim 5, wherein each storage node of the plurality of storage node is further programmed to:

receive a read instruction referencing a read snapshot, the read instruction including a read address;

if the read address is found in the index of any segment mapped in the segment table to at least one of (a) the read snapshot and (b) any snapshot preceding the read snapshot, returning last-written data from at least one of (a) and (b) corresponding to the read address;

if the read address is not found in the index of any segment mapped in the segment table to at least one of (a) the read snapshot and (b) any snapshot preceding the read snapshot, route the read instruction to a different storage node of the plurality of storage nodes that stores an ancestor snapshot of the read snapshot according to the hierarchical representation.

7. The system of claim 3, wherein the storage manager is further programmed to:

receive a roll back instruction to a selected snapshot for the storage volume;

in response to the roll back instruction, for each snapshot node in the hierarchical representation that is a descendent of the selected snapshot node and does not have a descendent having the clone classification, instruct a storage node of the plurality of storage nodes storing the each snapshot to free all segments mapped to a snapshot corresponding to the each snapshot node.

8. The system of claim 3, wherein each storage node of the plurality of storage nodes includes a physical disk, each segment in the segment table being mapped to a physical location on the physical disk.

9. The system of claim 3, further comprising:

a plurality of compute nodes in data communication with the network, the plurality of compute nodes programmed to execute applications that issue read and write instructions to the plurality of storage nodes, each read and write instruction referencing a snapshot represented by one of the snapshot nodes in the hierarchical representation.

10. The system of claim 9, wherein the storage manager is further programmed to:

receive a user instruction to create a new snapshot for the storage volume;

in response to the user instruction to create a new snapshot—

transmit a quiesce instruction to one or more of the plurality of compute nodes;

receive one or more notifications from the one or more of the plurality of compute nodes that there are no pending write instructions to the storage volume;

transmit one of the new snapshot instructions of the plurality of storage nodes in response to the one or more notifications.

11. A method comprising:

transmitting, by a storage manager comprising a processing device and one or more memory devices, a plurality of new snapshot instructions with respect to a storage volume, the plurality of new snapshot instructions including both of new principal snapshot instructions and at least one new clone instruction, each new snapshot instruction having a corresponding preceding snapshot node in a hierarchical representation of the storage volume; and

for each new snapshot instruction of the plurality of new snapshot instructions, adding a new snapshot node as a descendent to the corresponding preceding snapshot node of the each new snapshot instruction in the hierarchical representation, the new snapshot node for the at least one new clone instruction having a clone classification.

12. The method of claim 11, further comprising:

generating the hierarchical representation such that the hierarchical representation includes a tree having;

(a) multiple snapshot nodes that are descendants of an initial snapshot node;

(b) at least one snapshot node that has the clone classification and one or more descendent nodes of the at least one snapshot node that has the clone classification; and

(c) at least one snapshot node that has a direct descendent that has the clone classification and a direct descendent that does not have the clone classification.

13. The method of claim 11, further comprising:

receiving, by a storage node of a plurality of storage nodes coupled to the storage manager by a network, a new snapshot instruction of the plurality of snapshot instructions;

in response to receiving the new snapshot instruction—

allocating a new segment;

creating an entry in a segment table mapping the new segment to a new snapshot; and

writing data from subsequent write requests for the storage volume referencing the new snapshot to the new segment.

14. The method of claim 13, further comprising

receiving, by the storage node, a write instruction referencing a written snapshot, the write instruction including a write address;

identifying, by the storage node, a last-allocated segment corresponding to the written snapshot in the segment table;

writing, by the storage node, payload data from the write instruction to the last-allocated segment; and

writing, by the storage node, metadata to an index of the last-allocated segment corresponding to the written snapshot, the metadata including the write address.

15. The method of claim 14, wherein the storage node is a first storage node, the method further comprising:

transmitting, by the storage manager, one or more first new snapshot instructions of the plurality of snapshot instructions to the first storage node, the one or more first new snapshot instructions being principal snapshot instructions;

transmitting, by the storage manager, one or more second new snapshot instructions of the plurality of snapshot instructions to a second storage node of the one or plurality of storage nodes, the one or more second new snapshot instructions being clone snapshot instructions.

16. The method of claim 15, further comprising:

receiving, by the second storage node, a read instruction referencing a read snapshot, the read instruction including a read address;

determining that the read address is not found in the index of any segment mapped in the segment table to at least one of (a) the read snapshot and (b) any snapshot preceding the read snapshot

in response to determining that the read address is not found in the index of any segment mapped in the segment table to at least one of (a) the read snapshot and (b) any snapshot preceding the read snapshot, routing the read instruction to the first storage node.

17. The method of claim 13, further comprising:

receiving, by the storage manager, a roll back instruction to a selected snapshot for the storage volume;

in response to the roll back instruction, for each snapshot node in the hierarchical representation that is a descendent of the selected snapshot node and does not have a descendent having the clone classification, instructing, by the storage manager, a storage node of the plurality of storage nodes storing the each snapshot to free all segments mapped to a snapshot corresponding to the each snapshot node.

18. The method of claim 13, wherein each storage node of the plurality of storage nodes includes a physical disk, each segment in the segment table being mapped to a physical location on the physical disk.

19. The method of claim 13, further comprising:

providing a plurality of compute nodes in data communication with the plurality of storage nodes over the network;

issuing, by the plurality of compute nodes, read and write instructions to the plurality of storage nodes, each read and write instruction referencing a snapshot represented by one of the snapshot nodes in the hierarchical representation.

20. The method of claim 19, further comprising:

receiving, by the storage manager, a user instruction to create a new snapshot for the storage volume;

in response to the user instruction to create the new snapshot—

transmitting, by the storage manager, an instruction to suppress issuing further write requests to one or more of the plurality of compute nodes;

receiving, by the storage manager, one or more notifications from the one or more of the plurality of compute nodes that there are no pending write instructions to the storage volume;

transmitting, by the storage manager, a new snapshot instruction of the plurality of new snapshot instructions to one or more of the plurality of storage nodes in response to the one or more notifications.