Storage cluster

• US 9,934,089 B2
• Filed: 11/30/2015
• Issued: 04/03/2018
• Est. Priority Date: 06/04/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Hayes, John, Colgrove, John, Davis, John D.
• Primary Examiner(s): ELMORE, STEPHEN C
• Application Number:
  US14/954,757
• Publication Number:
  US 20160085628A1
• Time in Patent Office: 855 Days
• Field of Search: 711103, 711154, 711156, 711158, 711170, 711216, 713340, 714 62, 714 624, 714 632
• CPC Class Codes: G06F 11/1068 in sector programmable memories, e.g. flash disk G06F11/1072 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2201/845 Systems in which the redundancy can be transformed in increased performance
• CPC Class Codes: G06F 2212/7206 Reconfiguration of flash memory system
• CPC Class Codes: G06F 2212/7207 management of metadata or control data
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: H03M 13/154 Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial

Abstract

A plurality of storage nodes in a single chassis is provided. The plurality of storage nodes in the single chassis is configured to communicate together as a storage cluster. Each of the plurality of storage nodes includes nonvolatile solid-state memory for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes such that the plurality of storage nodes maintain the ability to read the user data, using erasure coding, despite a loss of two of the plurality of storage nodes. A plurality of compute nodes is included in the single chassis, each of the plurality of compute nodes is configured to communicate with the plurality of storage nodes. A method for accessing user data in a plurality of storage nodes having nonvolatile solid-state memory is also provided.

Claims

1. A method for storing user data in a storage cluster, comprising:

determining, by one of a plurality of authorities in a plurality of storage nodes of a storage cluster, which of a plurality of erasure coding schemes to use for a range of user data owned by the authority; and

storing a portion of user data across the plurality of storage nodes using the erasure coding scheme determined by the authority.

2. The method of claim 1, further comprising:

determining, in one of the plurality of storage nodes, which authority in which storage node of the storage cluster has ownership of the range of user data that includes the portion of user data that is arriving for storage in the storage cluster.

3. The method of claim 1, further comprising:

determining, in one or more of the plurality of storage nodes, for a further portion of user data, which authority in which storage node of the storage cluster has ownership of a range of user data that includes the further portion of user data;

determining, in the one or more of the plurality of storage nodes, that the authority that has the ownership of the range of user data that includes the further portion of user data is unreachable; and

determining, in the one or more of the plurality of storage nodes, where to locate the further portion of user data across the plurality of storage nodes in accordance with a redundant copy, in the plurality of storage nodes, of the authority that has the ownership of the range of user data that includes the further portion of user data.

4. The method of claim 1, further comprising:

calculating, in one of the plurality of storage nodes, a hash value for a user data segment;

mapping, in the one of the plurality of storage nodes, the hash value for the user data segment to an identifier of one of the plurality of authorities that has ownership of the user data segment; and

forwarding the user data segment to the authority that has the ownership of the user data segment, from the one of the plurality of storage nodes.

5. The method of claim 1, further comprising:

transferring an authority from one of the plurality of storage nodes to another of the plurality of storage nodes.

6. The method of claim 1, further comprising:

recovering user data with two or more erasure coding schemes in the storage cluster, with each of two or more of the plurality of authorities recovering user data according to an erasure coding scheme determined by that authority.

7. The method of claim 1, further comprising:

determining, in cooperation among at least a subset of the plurality of storage nodes, contents of a defective authority; and

recovering user data in the storage cluster, in accordance with the determined contents of the defective authority.

8. A storage cluster, with one or more processors therein configured to perform actions comprising:

deciding, in accordance with one of a plurality of authorities in a plurality of storage nodes of a storage cluster, which of a plurality of erasure coding schemes to use for a range of user data owned by the authority; and

writing a portion of user data across the plurality of storage nodes using the erasure coding scheme determined in accordance with the authority.

9. The method of claim 8, further comprising:

determining, in one of the plurality of storage nodes, which authority in which storage node of the storage cluster has ownership of the range of user data that includes the portion of user data that is arriving for storage in the storage cluster.

10. The method of claim 8, further comprising:

determining, in one or more of the plurality of storage nodes, for a further portion of user data, which authority in which storage node of the storage cluster has ownership of a range of user data that includes the further portion of user data;

determining, in the one or more of the plurality of storage nodes, that the authority that has the ownership of the range of user data that includes the further portion of user data is unreachable; and

determining, in the one or more of the plurality of storage nodes, where to locate the further portion of user data across the plurality of storage nodes in accordance with a redundant copy, in the plurality of storage nodes, of the authority that has the ownership of the range of user data that includes the further portion of user data.

11. The method of claim 8, further comprising:

calculating, in one of the plurality of storage nodes, a hash value for a user data segment;

mapping, in the one of the plurality of storage nodes, the hash value for the user data segment to an identifier of one of the plurality of authorities that has ownership of the user data segment; and

forwarding the user data segment to the authority that has the ownership of the user data segment, from the one of the plurality of storage nodes.

12. The method of claim 8, further comprising:

transferring an authority from one of the plurality of storage nodes to another of the plurality of storage nodes.

13. The method of claim 8, further comprising:

recovering user data with two or more erasure coding schemes in the storage cluster, with each of two or more of the plurality of authorities recovering user data according to an erasure coding scheme determined by that authority.

14. The method of claim 8, further comprising:

determining, in cooperation among at least a subset of the storage nodes, contents of a defective authority; and

recovering user data in the storage cluster, in accordance with the determined contents of the defective authority.

15. A tangible, non-transitory, computer-readable media having instructions thereupon which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

determining, by one of a plurality of authorities in a plurality of storage nodes of a storage cluster, which of a plurality of erasure coding schemes to use for a range of user data owned by the authority; and

storing a portion of user data across the plurality of storage nodes using the erasure coding scheme determined by the authority.

16. The method of claim 15, further comprising:

determining, in one of the plurality of storage nodes, which authority in which storage node of the storage cluster has ownership of the range of user data that includes the portion of user data that is arriving for storage in the storage cluster.

17. The method of claim 15, further comprising:

determining, in one or more of the plurality of storage nodes, for a further portion of user data, which authority in which storage node of the storage cluster has ownership of a range of user data that includes the further portion of user data;

determining, in the one or more of the plurality of storage nodes, that the authority that has the ownership of the range of user data that includes the further portion of user data is unreachable; and

determining, in the one or more of the plurality of storage nodes, where to locate the further portion of user data across the plurality of storage nodes in accordance with a redundant copy, in the plurality of storage nodes, of the authority that has the ownership of the range of user data that includes the further portion of user data.

18. The method of claim 15, further comprising:

calculating, in one of the plurality of storage nodes, a hash value for a user data segment;

mapping, in the one of the plurality of storage nodes, the hash value for the user data segment to an identifier of one of the plurality of authorities that has ownership of the user data segment; and

forwarding the user data segment to the authority that has the ownership of the user data segment, from the one of the plurality of storage nodes.

19. The method of claim 15, further comprising:

transferring an authority from one of the plurality of storage nodes to another of the plurality of storage nodes.

20. The method of claim 15, further comprising:

recovering user data with two or more erasure coding schemes in the storage cluster, with each of two or more of the plurality of authorities recovering user data according to an erasure coding scheme determined by that authority.