Die-level monitoring in a storage cluster

• US 10,579,474 B2
• Filed: 01/29/2018
• Issued: 03/03/2020
• Est. Priority Date: 08/07/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Davis, John D., Hayes, John, Kannan, Hari, Miladinovic, Nenad, Tan, Zhangxi
• Primary Examiner(s): Britt, Cynthia
• Application Number:
  US15/882,886
• Publication Number:
  US 20180165154A1
• Time in Patent Office: 764 Days
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/1048 using arrangements adapted for a specific error detection or correction feature
• CPC Class Codes: G06F 11/1068 in sector programmable memories, e.g. flash disk G06F11/1072 takes precedence
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 2212/7207 management of metadata or control data
• CPC Class Codes: G06F 2212/7208 Multiple device management, e.g. distributing data over multiple flash devices
• CPC Class Codes: G06F 2212/7211 Wear leveling
• CPC Class Codes: G11C 16/349 Arrangements for evaluating degradation, retention or wearout, e.g. by counting erase cycles
• CPC Class Codes: G11C 2029/0409 Online test
• CPC Class Codes: G11C 2029/0411 Online error correction
• CPC Class Codes: G11C 29/44 Indication or identification of errors, e.g. for repair
• CPC Class Codes: G11C 29/52 Protection of memory contents; Detection of errors in memory contents

Abstract

In some embodiments, a method for die-level monitoring is provided. The method includes distributing user data throughout a plurality of storage nodes through erasure coding, wherein the plurality of storage nodes are housed within a chassis that couples the storage nodes. Each of the storage nodes has a non-volatile solid-state storage with non-volatile memory and the user data is accessible via the erasure coding from a remainder of the storage nodes in event of two of the storage nodes being unreachable. The method includes producing diagnostic information that diagnoses the non-volatile memory on a basis of per package, per die, per plane, per block, or per page, the producing performed by each of the plurality of storage nodes. The method includes writing the diagnostic information to a memory in the storage cluster.

Claims

1. A plurality of storage nodes, comprising:

each of the plurality of storage nodes having non-volatile solid-state storage; and

each non-volatile solid-state storage having a controller configured to track bit errors of solid-state memory in the non-volatile solid-state storage, wherein diagnostic information for each non-volatile solid-state storage is based at least in part on the bit errors.

2. The plurality of storage nodes of claim 1, further comprising

the plurality of storage nodes configured to derive characterization information from the diagnostic information, the characterization information indicating at least one change, prediction or trend in the non-volatile solid-state storage.

3. The plurality of storage nodes of claim 1, further comprising

the plurality of storage nodes configured to report the diagnostic information via a network.

4. The plurality of storage nodes of claim 1, further comprising

non-volatile memory included in the non-volatile solid-state storage;

each non-volatile solid-state storage having a controller, the controller configured to track bit errors of the non-volatile memory;

each of the plurality of storage nodes having a processor, and

the controller of each non-volatile solid-state storage configured to forward information regarding the bit errors to each processor.

5. The plurality of storage nodes of claim 1, further comprising:

each of the plurality of storage nodes configured to direct a portion of non-volatile memory of non-volatile solid-state storage to change one of;

a variable write time, a variable write voltage, a variable read time, a variable reference voltage, a variable reference current, or a variable number of reads, based on the diagnostic information.

6. The plurality of storage nodes of claim 1, further comprising:

a wear list, coupled to the plurality of storage nodes and configured to categorize blocks of the non-volatile solid-state storage by wear level;

a retention list, coupled to the plurality of storage nodes and configured to track data retention in the non-volatile solid-state storage; and

the plurality of storage nodes configured to update the wear list responsive to the diagnostic information.

7. A storage cluster, comprising

a plurality of storage nodes within a chassis;

each of the plurality of storage nodes having a controller with an error correction block, wherein diagnostic information is based on a result of feedback from the error correction block.

8. The storage cluster of claim 7, further comprising:

each of the plurality of storage nodes having a controller with an error correction block configured to perform error correction on a data stripe that includes reads of non-volatile memory across the plurality of storage nodes, wherein the diagnostic information is based on feedback from the error correction block of each of the plurality of storage nodes.

9. The storage cluster of claim 7, further comprising:

each of the plurality of storage nodes configured to determine relative wear of blocks; and

each of the plurality of storage nodes configured to direct a write to a second block, based on a determination that the second block has a lower wear than a first block according to the diagnostic information.

10. The storage cluster of claim 7, further comprising:

each of the plurality of storage nodes configured to direct a portion of non-volatile memory to change a read reference voltage or a read reference current, responsive to a determination that the portion of the non-volatile memory has one of a greater wear, a greater retention time, a greater read time, or a greater write time, than a further portion of the non-volatile memory according to the diagnostic information.

11. The storage cluster of claim 7, further comprising:

each of the plurality of storage nodes configured to direct a portion of non-volatile memory to change from multilevel cell (MLC) operation to single level cell (SLC) operation, responsive to a determination that the portion of the non-volatile memory has a greater wear than a further portion of the non-volatile memory according to the diagnostic information.

12. The storage cluster of claim 7, further comprising:

the plurality of storage nodes configured to track trends or changes of non-volatile memory based on the diagnostic information; and

the plurality of storage nodes configured to derive characterization information from the diagnostic information pertaining to the non-volatile memory, and from one of application data or meta data, pertaining to the user data.