Distributed scalable storage

• US 10,678,461 B2
• Filed: 06/07/2018
• Issued: 06/09/2020
• Est. Priority Date: 06/07/2018
• Status: Active Grant

• Current Assignee: VAST Data Ltd.
• Original Assignee: VAST Data Ltd.
• Inventors: Hallak, Renen, Levy, Asaf, Goren, Avi, Vaknin, Yogev, Turin, Alex
• Primary Examiner(s): Peugh, Brian R
• Application Number:
  US16/002,676
• Publication Number:
  US 20190377490A1
• Time in Patent Office: 733 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2237 Vectors, bitmaps or matrices
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A storage system. The storage system includes a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers; and a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is configured to receive an access command and to execute the access command based on the elements.

Claims

1. A storage system, comprising:

a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers; and

a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is configured to receive an access command and to execute the access command based on the elements, wherein the DNodes further include a plurality of random access memory (RAM) devices, wherein executing the access command further includes locking, by one of the CNodes, a portion of the plurality of RAM devices using at least one remote direct memory access compare-and-swap operation.

2. The storage system of claim 1, wherein the plurality of pointers includes a pointer to each of the data blocks, wherein execution of the access command includes identifying a portion of one of the DNodes based on the plurality of pointers and performing the access command on the identified portion.

3. The storage system of claim 1, wherein the plurality of pointers further includes at least one handle, wherein each handle is a unique identifier of one of the elements.

4. The storage system of claim 3, wherein the elements include a plurality of metadata blocks, wherein the attributes are stored in the metadata blocks, wherein each element has an element tree including a set of the metadata blocks.

5. The storage system of claim 4, each element tree having a root, wherein the roots of the element trees are stored in a global hash table, wherein the global hash table is distributed among the plurality of DNodes.

6. The storage system of claim 4, wherein the DNodes further include a plurality of 3D Xpoint devices and a plurality of Flash memory devices, wherein each element tree is stored partially in at least one of the 3D Xpoint devices and partially in at least one of the Flash memory devices.

7. The storage system of claim 6, each 3D Xpoint device having a write buffer including an unused portion, wherein each CNode is further configured to temporarily write each of the data blocks to a respective 3D Xpoint device of the 3D Xpoint devices, wherein each CNode is further configured to write one of the temporarily written data blocks to one of the Flash memory devices when the unused portion of the write buffer of the respective 3D Xpoint device is below a threshold.

8. The storage system of claim 4, wherein executing the access command includes locking at least one of the metadata blocks when the access command is a write command.

9. The storage system of claim 8, wherein the at least one of the metadata blocks is not locked by a central lock manager.

10. The storage system of claim 4, wherein executing the access command does not include locking a portion of the DNodes when the access command is a read command.

11. The storage system of claim 1, wherein at least one of the data blocks is stored in at least one of the elements when the at least one of the data blocks has a size below a threshold size.

12. The storage system of claim 1, wherein the DNodes support a protocol, wherein each CNode is further configured to translate the received access command based on the protocol supported by the DNodes.

13. A storage system, comprising:

a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers, wherein the plurality of pointers further includes at least one handle, wherein each handle is a unique identifier of one of the elements, wherein the elements include a plurality of metadata blocks, wherein the attributes are stored in the metadata blocks, wherein each element has an element tree including a set of the metadata blocks, wherein the DNodes further include a plurality of 3D Xpoint devices and a plurality of Flash memory devices, wherein each element tree is stored partially in at least one of the 3D Xpoint devices and partially in at least one of the Flash memory devices; and

a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is configured to receive an access command and to execute the access command based on the elements.

14. The storage system of claim 13, each 3D Xpoint device having a write buffer including an unused portion, wherein each CNode is further configured to temporarily write each of the data blocks to a respective 3D Xpoint device of the 3D Xpoint devices, wherein each CNode is further configured to write one of the temporarily written data blocks to one of the Flash memory devices when the unused portion of the write buffer of the respective 3D Xpoint device is below a threshold.

15. A storage system, comprising:

a plurality of storage nodes (DNodes), wherein the DNodes are configured to store a plurality of elements and a plurality of data blocks, wherein each element is a persistent metadata structure, wherein at least one of the elements store at least one attribute, wherein the at least one attribute includes a plurality of pointers, wherein the plurality of pointers further includes at least one handle, wherein each handle is a unique identifier of one of the elements, wherein the elements include a plurality of metadata blocks, wherein the attributes are stored in the metadata blocks, wherein each element has an element tree including a set of the metadata blocks, wherein the DNodes further include a plurality of 3D Xpoint devices and a plurality of Flash memory devices, wherein each element tree is stored partially in at least one of the 3D Xpoint devices and partially in at least one of the Flash memory devices; and

a plurality of compute nodes (CNodes), wherein each CNode has access to each of the DNodes, wherein each CNode is configured to receive an access command and to execute the access command based on the elements, wherein executing the access command includes locking at least one of the metadata blocks when the access command is a write command.

16. The storage system of claim 15, wherein the at least one of the metadata blocks is not locked by a central lock manager.

17. The storage system of claim 15, wherein executing the access command does not include locking a portion of the DNodes when the access command is a read command.

18. The storage system of claim 15, wherein the plurality of pointers includes a pointer to each of the data blocks, wherein execution of the access command includes identifying a portion of one of the DNodes based on the plurality of pointers and performing the access command on the identified portion.

19. The storage system of claim 15, wherein at least one of the data blocks is stored in at least one of the elements when the at least one of the data blocks has a size below a threshold size.

20. The storage system of claim 15, wherein the DNodes support a protocol, wherein each CNode is further configured to translate the received access command based on the protocol supported by the DNodes.