Mechanism for persisting messages in a storage system

• US 10,430,306 B2
• Filed: 10/24/2016
• Issued: 10/01/2019
• Est. Priority Date: 06/04/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Davis, John, Gold, Brian, Gupta, Shantanu, Hayes, John, Lee, Robert, Ostrovsky, Igor, Sears, Rusty
• Primary Examiner(s): Cardwell, Eric
• Application Number:
  US15/332,321
• Publication Number:
  US 20170075782A1
• Time in Patent Office: 1,072 Days
• CPC Class Codes: G06F 11/1016 Error in accessing a memory location, i.e. addressing error
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 12/0638 Combination of memories, e.g. ROM and RAM such as to permit replacement or supplementing of words in one module by words in another module address formation of the next microinstruction G06F9/26; masking faults in memories by using spares or by reconfiguring G11C29/70
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2212/205 Hybrid memory, e.g. using both volatile and non-volatile memory
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

A plurality of storage nodes is provided. Each of the plurality of storage nodes includes nonvolatile solid-state memory for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes such that the plurality of storage nodes maintain the ability to read the user data, using erasure coding, despite a loss of two of the plurality of storage nodes. The plurality of storage nodes is configured to initiate an action based on the redundant copies of the metadata, responsive to achieving a level of redundancy for the redundant copies of the metadata. A method for accessing user data in a plurality of storage nodes having nonvolatile solid-state memory is also provided.

Claims

1. A storage system, comprising:

a plurality of storage nodes;

the plurality of storage nodes configurable to distribute redundant copies of a message and write the redundant copies of the message to NVRAM (non-volatile random-access memory);

the plurality of storage nodes configurable to write the redundant copies of the message from the NVRAM to flash memory; and

the plurality of storage nodes configurable to open the redundant copies of the message, responsive to restoration of power after power loss, and cooperate to determine an action to carry to completion, and wherein each of the plurality of storage nodes is configurable to have a plurality of authorities with each authority in memory in the storage system, supported by a controller or processor in the storage system and controlling how and where data is written in the storage system according to a range of data identifiers for that authority, and controlling how and where data is written in the storage system according to the redundant copies of messages between sending authorities and receiving authorities.

2. The storage system of claim 1, further comprising:

the plurality of storage nodes configurable to act upon the message in a target storage node, responsive to achieving a redundancy level with the redundant copies of the message.

3. The storage system of claim 1, wherein:

each of the plurality of storage nodes has one or more storage units with a portion of the NVRAM, flash memory and energy reserve; and

the plurality of storage nodes are configured to vote, achieve consensus, or witness the redundant copies of the message, to cooperate to determine the action to carry to completion.

4. The storage system of claim 1, wherein the action to carry to completion is one of writing a data stripe, reading a data stripe, or creating a file with directory update.

5. The storage system of claim 1, wherein the message is one of a request to write data, a request to read data, a request to lock or unlock a file, a change in permission of a file, or an update to a directory structure.

6. The storage system of claim 1, further comprising:

the plurality of storage nodes configurable to derive a sequence of actions from the message and store the sequence of actions as metadata in one of the plurality of storage nodes, with replication of the metadata in further ones of the plurality of storage nodes.

7. The storage system of claim 1, further comprising:

the plurality of storage nodes configurable to restore redundancy of a record relating to task completion, wherein the plurality of storage nodes is configurable to cooperate to determine the action to carry to completion is based on the restored redundancy of the record relating to task completion.

8. A storage cluster, comprising:

a plurality of storage nodes, configurable to write data stripes across the plurality of storage nodes;

each of at least a subset of the plurality of storage nodes configurable to write a redundant copy of a message to NVRAM (non-volatile random-access memory) having an energy reserve;

each of the at least a subset of the plurality of storage nodes configurable to write the redundant copy of the message from the NVRAM to flash memory; and

the plurality of storage nodes configurable to access the redundant copies of the message and determine an action to carry to completion based on the redundant copies of the message, responsive to restart after the loss of power, and wherein each of the plurality of storage nodes is configurable to have a plurality of authorities with each authority in memory in the storage cluster, supported by a controller or processor in the storage cluster and controlling how and where data is written in the storage cluster according to a range of data identifiers for that authority, and controlling how and where data is written in the storage system according to such redundant copies of messages between sending authorities and receiving authorities.

9. The storage cluster of claim 8, further comprising:

the plurality of storage nodes configurable so that a target storage node acts upon the message, responsive to verification of a specified redundancy level for the redundant copies of the message.

10. The storage cluster of claim 8, further comprising:

a plurality of storage units, each having a portion of the NVRAM, flash memory and energy reserve, each of the plurality of storage nodes having at least one of the plurality of storage units; and

the plurality of storage nodes having voting logic, consensus logic, or witnessing, for the redundant copies of the message, to determine the action to carry to completion based on the redundant copies of the message.

11. The storage cluster of claim 8, wherein:

the message is a request to write data; and

the action to carry to completion includes writing a data stripe.

12. The storage cluster of claim 8, wherein:

the message is a request to read data; and

the action to carry to completion includes reading a data stripe.

13. The storage cluster of claim 8, further comprising:

the plurality of storage nodes configurable to form a sequence of actions based on the message; and

the plurality of storage nodes configurable to replicate metadata relating to the sequence of actions, in the plurality of storage nodes.

14. The storage cluster of claim 8, further comprising:

the plurality of storage nodes configurable to store redundant copies of a record relating to task completion, responsive to the task completion; and

the plurality of storage nodes configurable to restore redundancy of the record relating to task completion, wherein the action to carry to completion is further based on the restored redundancy of the record relating to task completion.

15. A method for persisting messages in a storage system, executed by the storage system, comprising:

distributing redundant copies of a message across a plurality of storage nodes in a storage cluster;

writing the redundant copies of the message to NVRAM (non-volatile random-access memory);

writing the redundant copies of the message from the NVRAM to flash memory, responsive to loss of power;

accessing the redundant copies of the message, responsive to restart after the loss of power; and

determining an action to carry to completion, based on the redundant copies of the message, and wherein each of the plurality of storage nodes is configurable to have a plurality of authorities with each authority in memory in the storage cluster, supported by a controller or processor in the storage cluster and controlling how and where data is written in the storage cluster according to a range of data identifiers for that authority, and controlling how and where data is written in the storage system according to such redundant copies of messages between sending authorities and receiving authorities.

16. The method of claim 15, further comprising:

performing an action in a target one of the plurality of storage nodes, based on achieving a specified redundancy level with the redundant copies of the message.

17. The method of claim 15, wherein the determining the action to carry to completion comprises:

witnessing or voting on the redundant copies of the message.

18. The method of claim 15, wherein determining the action to carry to completion comprises determining to read or write a data stripe or update a directory.

19. The method of claim 15, further comprising:

determining a sequence of actions based on the message; and

replicating metadata relating to the sequence of actions, in the plurality of storage nodes.

20. The method of claim 15, further comprising:

restoring redundancy of a record relating to task completion, wherein the determining the action to carry to completion is further based on the restoring the redundancy of the record relating to task completion.