Managing Data in a Storage System

• US 20170131913A1
• Filed: 01/25/2017
• Published: 05/11/2017
• Est. Priority Date: 05/20/2013
• Status: Active Grant

• Current Assignee: Kaminario Limited
• Original Assignee: Kaminario Limited
• Inventors: Gordon, Eyal, Koren, Benny, Atzmony, Yedidia, Tal, Doron, Fienblit, Shachar, Steinberg, Ilan
• Granted Patent: US 9,965,217 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0671 In-line storage system
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

According to an aspect of the presently disclosed subject matter, there is provided a system for managing data in a storage system, the system including a storage layer which provides storage resource, and a snapshot layer that includes: a volume-version data structure, a chunk-version data structure and a IO handler.

Claims

1. A method of managing data in a storage system, comprising:

maintaining a volume-version data structure including LU volume entries for a LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept;

maintaining a chunk-version data structure that is uniquely associated with a volume ID and a logical chunk address combination;

including in the chunk-version data structure one or more chunk-version data elements, each chunk version data element including a timestamp at a corresponding write to the respective logical chunk address in the respective volume and a pointer to storage resources associated with the chunk-version data element.

2. The method according to claim 1, wherein responsive to an instruction to create a new snapshot volume where a source of the snapshot is the LU volume or a snapshot volume that is associated with the LU volume, updating the volume-version data structure with:

a volume ID reference for the new snapshot-volume, an indication of a connection between the new snapshot-volume and the source, and a timestamp for the snapshot operation.

3. The method according to claim 1, further comprising responsive to receiving a write command in the storage system:

determining with which snapshot layer chunk the write command is associated; and

determining whether there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated.

4. The method according to claim 3, wherein in case there is no existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, generating a new chunk version data structure that is uniquely associated with the volume ID and with a logical chunk address of the snapshot layer chunk with which the write command is associated, and generating in the chunk-version data structure a chunk-version data element including a current timestamp and a pointer to storage resources that were allocated for this chunk-version data element.

5. The method according to claim 3, wherein in case there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, determining whether a new chunk version data element needs to be generated or whether an existing chunk version data element can be overwritten.

6. The method according to claim 5, wherein in case it is determined that a new chunk version data element needs to be generated, allocating storage resources for the new chunk version data element.

7. The method according to claim 6, further comprising requesting allocation of storage resources for the new chunk version data element from a free storage layer resources list.

8. The method according to claim 5, wherein in case it is determined that there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, and that an existing chunk version data element can be overwritten, writing write data of the write command in storage resources that were allocated to a last chunk version data element in the chunk version data structure.

9. The method according to claim 8, wherein in response to writing the write data of the write command in the storage resources that were allocated to the last chunk version data element in the chunk version data structure updating the timestamp of the last chunk version data element with a current timestamp.

10. The method according to claim 5, wherein determining whether a new chunk version data element needs to be generated or whether an existing chunk version data element can be overwritten, comprises:

reading a timestamp from a last chunk-version data element in the existing chunk-version data structure; and

if there is no snapshot operation whose source is the volume referenced in the write command, or if all snapshot operations whose source is the volume referenced in the write command have a timestamp that is earlier than or equal to the timestamp in the last chunk-version data element, determining that the last chunk-version data element in the existing chunk-version data structure can be overwritten.

54. The method according to claim 1, wherein further in response to the instruction to create a new snapshot volume suspending IO to the source volume and incrementing a timestamp for the snapshot operation, and once the snapshot operation is complete, enabling IOs to the source volume to resume.

55. The method according to 4, wherein the timestamp is a logical timestamp, and wherein the logical timestamp is incremented as a result of a volume create, a snapshot create or a revert operation.

11. A storage system, comprising:

a storage layer providing storage resources;

a snapshot layer comprising;

a volume-version data structure including LU volume entries for an LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept;

a chunk-version data structure that is uniquely associated with a volume ID and a logical chunk address combination;

a IO handler that is configured to include in the chunk-version data structure one or more chunk-version data elements, each chunk version data element including a timestamp at a corresponding write to the respective logical chunk address in the respective volume and a pointer to storage resources associated with the chunk-version data element.

12. The storage system according to claim 11, wherein responsive to an instruction to create a new snapshot volume where a source of the snapshot is the LU volume or a snapshot volume that is associated with the LU volume, the IO handler is configured to update the volume-version data structure with:

a volume ID reference for the new snapshot-volume, an indication of a connection between the new snapshot-volume and the source, and a timestamp for the snapshot operation.

13. The storage system according to claim 11, further comprising responsive to receiving a write command in the storage system:

the IO handler is configured to determine with which snapshot layer chunk the write command is associated; and

the IO handler is configured to determine whether there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated.

14. The storage system according to claim 13, wherein in case there is no existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, the IO handler is configured to generate a new chunk version data structure that is uniquely associated with the volume ID and with a logical chunk address of the snapshot layer chunk with which the write command is associated, and to generate in the chunk-version data structure a chunk-version data element including a current timestamp and a pointer to storage resources that were allocated for this chunk-version data element.

15. The storage system according to claim 13, wherein in case there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, the IO handler is configured to determine whether a new chunk version data element needs to be generated or whether an existing chunk version data element can be overwritten.

16. The storage system according to claim 15, further comprising a free storage resources list, and wherein in case it is determined that a new chunk version data element needs to be generated, the IO handler is configured to request from the free storage resources list allocation of storage resources for the new chunk version data element.

17. The storage system according to claim 14, wherein the IO handler is responsive to determining there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, and that an existing chunk version data element can be overwritten, for writing write data of the write command in storage resources that were allocated to last chunk version data element in the chunk version data structure and for updating the timestamp of the last chunk version data element with a current timestamp.

18. The storage system according to claim 14, wherein the IO handler is configured to determine whether a new chunk version data element needs to be generated or whether an existing chunk version data element can be overwritten, by:

reading a timestamp from a last chunk-version data element in the existing chunk-version data structure; and

if there is no snapshot operation whose source is the volume referenced in the write command, or if all snapshot operations whose source is the volume referenced in the write command have a timestamp that is earlier than or equal to the timestamp in the last chunk-version data element, the IO handler is configured to determine that the last chunk-version data element in the existing chunk-version data structure can be overwritten.

19. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform a method of managing data in a storage system, comprising:

maintaining a volume-version data structure including LU volume entries for an LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept;

maintaining a chunk-version data structure that is uniquely associated with a volume ID and a logical chunk address combination; and

including in the chunk-version data structure one or more chunk-version data elements, each chunk version data element including a timestamp at a corresponding write to the respective logical chunk address in the respective volume and a pointer to storage resources associated with the chunk-version data element.

20. A computer program product comprising a computer useable medium having computer readable program code embodied therein for managing data in a storage system, the computer program product comprising:

computer readable program code for causing the computer to maintain a volume-version data structure including LU volume entries for an LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept;

computer readable program code for causing the computer to maintain a chunk-version data structure that is uniquely associated with a volume ID and a logical chunk address combination; and

computer readable program code for causing the computer to include in the chunk-version data structure one or more chunk-version data elements, each chunk version data element including a timestamp at a corresponding write to the respective logical chunk address in the respective volume and a pointer to storage resources associated with the chunk-version data element.

21. A method of handling writes in a storage system, comprising:

in response to receiving a write command determining with which snapshot layer chunk the write command is associated, and whether there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated;

in case the write command is associated with an existing chunk-version data structure, reading a timestamp from a last chunk-version data element in the existing chunk-version data structure, and if there is a snapshot operation whose source is a volume referenced in the write command and this snapshot operation is associated with a timestamp that is later than a timestamp of the last chunk-version data element, maintaining the last chunk-version data element and adding to the existing chunk-version data structure a new chunk version data element for the write command.

22. The method according to claim 21, wherein adding to the existing chunk-version data structure a new chunk version data element includes adding a reference to storage resources associated with the new chunk-version data element.

23. The method according to claim 22, further comprising requesting allocation of storage resources for the new chunk version data element from a free storage layer resources list.

24. The method according to claim 21, further comprising maintaining a volume-version data structure including volume entries for a LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept.

25. The method according to claim 24, further comprising maintaining chunk-version data structures, each of which is associated with a volume ID and a logical chunk address, and each of which includes one or more chunk version data elements, wherein each chunk version data element includes a timestamp at a corresponding write to the respective logical chunk address and a pointer to storage resources associated with the chunk-version data element.

26. The method according to claim 25, wherein in case there is no existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, generating a new chunk version data structure that is uniquely associated with the volume ID and with a logical chunk address of the snapshot layer chunk with which the write command is associated, and generating in the chunk-version data structure a new chunk-version data element including a timestamp at the write command and a pointer to storage resources that were allocated for this chunk-version data element.

27. The method according to claim 25, in case there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, and there is an existing chunk version data element in the existing chunk version data structure that can be overwritten, writing write data of the write command in storage resources that were allocated to a last chunk version data element in the existing chunk version data structure.

28. The method according to claim 27, comprising determining that the existing chunk version data element in the existing chunk version data structure can be overwritten when the write command is associated with an existing chunk-version data structure, and the timestamp of the last chunk-version data element in the existing chunk-version data structure is later than or equal to the timestamp of a snapshot operation in which the source volume is the volume referenced in the write command.

29. A storage system, comprising:

a storage layer providing storage resources;

a snapshot layer comprising;

a volume-version data structure;

a chunk-version data structure;

a IO handler responsive to receiving a write command in the snapshot layer for determining with which snapshot layer chunk the write command is associated, and whether there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, and

in case the write command is associated with an existing chunk-version data structure, the IO handler is configured to read a timestamp from a last chunk-version data element in the existing chunk-version data structure, and

if there is a snapshot operation whose source is a volume referenced in the write command and this snapshot operation is associated with a timestamp that is later than a timestamp of the last chunk-version data element, the IO handler is configured to maintain the last chunk-version data element and to add to the existing chunk-version data structure a new chunk version data element for the write command.

30. The storage system according to claim 29, wherein the IO handler is configured to including in the new chunk version data element a reference to storage resources which are associated with the new chunk-version data element.

31. The storage system according to claim 30, further comprising a free storage resources list, and wherein in case it is determined that a new chunk version data element needs to be generated, the IO handler is configured to request from the free storage resources list allocation of storage resources for the new chunk version data element.

32. The storage system according to claim 29, wherein the volume-version data structure includes volume entries for a LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept.

33. The storage system according to claim 32, wherein the chunk-version data structure is associated with a volume ID and a logical chunk address, and includes one or more chunk version data elements, wherein each chunk version data element includes a timestamp at a corresponding write to the respective logical chunk address and a pointer to storage resources associated with the chunk-version data element.

34. The storage system according to claim 33, wherein in case the IO handler determines that there is no existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, the IO handler is configured to generate a new chunk version data structure that is uniquely associated with the volume ID and with a logical chunk address of the snapshot layer chunk with which the write command is associated, and to generate in the chunk-version data structure a new chunk-version data element including a timestamp at the write command and a pointer to storage resources that were allocated for this chunk-version data element.

35. The storage system according to claim 33, in case the IO handler determines that there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated, and that there is an existing chunk version data element in the existing chunk version data structure that can be overwritten, the IO handler is configured to write write-data of the write command in storage resources that were allocated to a last chunk version data element in the existing chunk version data structure.

36. The storage system according to claim 35, wherein the IO handler is configured to determine that the existing chunk version data element in the existing chunk version data structure can be overwritten when the write command is associated with an existing chunk-version data structure, and the timestamp of the last chunk-version data element in the existing chunk-version data structure is later than or equal to the timestamp of a snapshot operation in which the source volume is the volume referenced in the write command.

37. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform a method of handling writes in a storage system, comprising:

in response to receiving a write command determining with which snapshot layer chunk the write command is associated, and whether there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated;

in case the write command is associated with an existing chunk-version data structure, reading a timestamp from a last chunk-version data element in the existing chunk-version data structure, and if there is a snapshot operation whose source is a volume referenced in the write command and this snapshot operation is associated with a timestamp that is later than a timestamp of the last chunk-version data element, maintaining the last chunk-version data element and adding to the existing chunk-version data structure a new chunk version data element for the write command.

38. A computer program product comprising a computer useable medium having computer readable program code embodied therein for handling writes in a storage system, the computer program product comprising:

computer readable program code that is responsive to receiving a write command in the storage system for causing the computer to determine with which snapshot layer chunk the write command is associated, and to determine whether there is an existing chunk version data structure which is associated with the snapshot layer chunk with which the write command is associated;

computer readable program code for causing the computer to determine whether the write command is associated with an existing chunk-version data structure, and in case the write command is associated with an existing chunk-version data structure, the computer readable program code is configured to cause a timestamp from a last chunk-version data element in the existing chunk-version data structure to be read, and if there is a snapshot operation whose source is a volume referenced in the write command and this snapshot operation is associated with a timestamp that is later than a timestamp of the last chunk-version data element, the computer readable program code is configured to cause the last chunk-version data element to maintained and a new chunk version data element for the write command to be added to the existing chunk-version data structure.

44. The method according to claim 38, further comprising maintaining a volume-version data structure including volume entries for a LU volume and for snapshot volumes associated with the LU volume, wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept.

45. The method according to claim 44, further comprising maintaining chunk-version data structures, each of which is associated with a volume ID and a logical chunk address, and each of which includes one or more chunk version data elements, wherein each chunk version data element includes a timestamp of a corresponding write to the respective logical chunk address and a pointer to storage resources associated with the chunk-version data element.

39. A method of handling reads in a storage system comprising:

(a) in response to receiving a read command at the storage system determining with which volume and with which snapshot layer chunk the read command is associated, initializing a read pointer with respective volume ID and logical chunk address, and initializing a time-marker according to a timestamp at receipt of the read command;

(b) determining whether there is a chunk-version data structure that is associated with a current read pointer;

(c1) if the condition in operation (b) is not met, determining whether there is a snapshot operation where the volume that is referenced in the write pointer is a target volume and whether the snapshot operation is associated with a timestamp that is not greater than the current time-marker; and

(d1) if the conditions in operation (c) are met updating the read pointer with a volume ID of a source volume of the snapshot operation, updating the time-marker with the timestamp for the snapshot operation—

1, and returning to operation (b).

40. The method according to claim 39, further comprising:

(c2) if the condition in operation (b) is met, determining whether in the chunk version data structure there is a chunk version data element whose timestamp is not greater than a current time-marker; and

(e1) if the condition in operation (c2) is met reading data from the storage resource referenced in a last chunk version element of the chunk version data structure.

41. The method according to claim 40, wherein in response to implementing operation (e1):

(f) determining whether a search was conducted for all logical blocks referenced in the read command;

(g1) and if the condition in operation (f) is met responding to the read command with data obtain in operation (e1) and including invalid data indication(s) for data blocks referenced in the read command for which valid data could not be obtained.

42. The method according to claim 40, wherein in case the condition in operation (c2) is not met, returning to operation (c1).

43. The method according to claim 41, wherein in case the condition in operation (f) is not met, returning to operation (c1).

50. The method according to claim 41, wherein in case the condition in operation (f) is not met, the IO handler is configured to return to operation (c1).

46. A storage system, comprising:

a storage layer providing storage resources;

a snapshot layer comprising;

a volume-version data structure;

a chunk-version data structure;

a IO handler responsive to receiving a read comment for;

(a) determining with which volume and with which snapshot layer chunk the read command is associated, initializing a read pointer with respective volume ID and logical chunk address, and initializing a time-marker according to a timestamp of the read command;

(b) determining whether there is a chunk-version data structure that is associated with a current read pointer;

(c1) if the condition in operation (b) is not met, determining whether there is a snapshot operation where the volume that is referenced in the write pointer is a target volume and whether the snapshot operation is associated with a timestamp that is not greater than the current time-marker; and

(d1) if the conditions in operation (c) are met the IO under is configured for;

updating the read pointer with a volume ID of a source volume of the snapshot operation, updating the time-marker with the timestamp for the snapshot operation—

1, and returning to operation (b).

47. The storage system according to claim 46, wherein in case the condition in operation (b) is met the IO handler is configured for:

(c2) determining whether in the chunk version data structure there is a chunk version data element whose timestamp is not greater than a current time-marker; and

(e1) if the condition in operation (c2) is met the IO handler is configured for reading data from the storage resource referenced in a last chunk version element of the chunk version data structure.

48. The storage system according to claim 47, wherein in response to implementing operation (e1), the IO handler is configured for:

(f) determining whether a search was conducted for all logical blocks referenced in the read command;

(g1) and if the condition in operation (f) is met, the IO handler is configured for responding to the read command with data obtain in operation (e1) and including invalid data indication(s) for data blocks referenced in the read command for which valid data could not be obtained.

49. The storage system according to claim 47, wherein in case the condition in operation (c2) is not met, the IO handler is configured to return to operation (c1).

51. The storage system according to claim 46, wherein the volume-version data structure includes volume entries for a LU volume and for snapshot volumes associated with the LU volume, and wherein for each volume a respective volume ID is kept, and for each snapshot a timestamp for the snapshot operation and an indication of a connection between a respective target volume and a source volume are kept.

52. The method according to claim 51, wherein each one of the chunk-version data structures is associated with a volume ID and a logical chunk address, and includes one or more chunk version data elements, wherein each chunk version data element includes a timestamp of a corresponding write to the respective logical chunk address and a pointer to storage resources associated with the chunk-version data element.

53. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform a method of handling reads in a storage system comprising:

(a) in response to receiving a read command at the storage system determining with which volume and with which snapshot layer chunk the read command is associated, initializing a read pointer with respective volume ID and logical chunk address, and initializing a time-marker according to a timestamp of the read command;

(b) determining whether there is a chunk-version data structure that is associated with a current read pointer;

(c1) if the condition in operation (b) is not met, determining whether there is a snapshot operation where the volume that is referenced in the write pointer is a target volume and whether the snapshot operation is associated with a timestamp that is not greater than the current time-marker; and

(d1) if the conditions in operation (c) are met updating the read pointer with a volume ID of a source volume of the snapshot operation, updating the time-marker with the timestamp for the snapshot operation—

1, and returning to operation (b).