Detailed memory device statistics with drive write location determination

• US 10,216,434 B2
• Filed: 11/14/2016
• Issued: 02/26/2019
• Est. Priority Date: 12/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K.
• Primary Examiner(s): Eskandarnia, Arvin
• Application Number:
  US15/350,677
• Publication Number:
  US 20170195420A1
• Time in Patent Office: 834 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 12/0684 with feedback, e.g. presence or absence of unit detected by addressing, overflow detection
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0888 using selective caching, e.g. bypass
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0616 in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0652 : Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0653 : Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 : Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 : using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 43/0852 : Delays
• CPC Class Codes: H04L 43/0876 : Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/0888 : Throughput
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Method and systems for selecting memory resources in a dispersed storage network (DSN) include a processing module obtaining deep statistics for one or more memory device of at least some storage units of a plurality of storage units of the DSN. The processing module also determines a performance level of the memory device based on the associated deep statistics of at least the memory device and further determines whether to access a memory device based on the associated performance level of the memory device. For example, the processing module indicates not to access the memory device when the performance level of the memory device is less than a minimum performance threshold level and selects another memory device associated with a favorable performance level. In addition to, the processing module can generate a new DSN address for new data storage where the new DSN address is associated with the another memory device.

Claims

1. A method of selecting memory resources in a dispersed storage network, the dispersed storage network including a plurality of dispersed storage units, the plurality of dispersed storage units each including a respective plurality of memories, the method comprising:

obtaining respective deep statistics for the respective plurality of memories of one or more of the plurality of dispersed storage units;

determining a respective performance level of one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units; and

determining whether to access one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

2. The method of claim 1, wherein the respective deep statistics pertain to respective ranges of slice names assigned to the respective plurality of memories of the one or more of the plurality of dispersed storage units.

3. The method of claim 1, wherein the respective deep statistics include one or more of a memory device identifier, a slice name range, a throughput level, a process loading level indicator (IOPS), a number of read operations per unit of time, a number of write operations per unit of time, a number of delete operations per unit of time, an error count, an error count percentage, a number of errors per unit of time, a read latency level, a write latency level, a storage capacity utilization level, a storage capacity level, a number of requests received per unit of time, a rebuild activity rate, a rebalancing activity rate, and a migration activity rate.

4. The method of claim 1, wherein the step of obtaining respective deep statistics for the respective plurality of memories of one or more of the plurality of dispersed storage units includes one or more of issuing a deep statistics request, interpreting a deep statistics response, and extracting the respective deep statistics from a recovered data object stored as at least one set of data object slices.

5. The method of claim 1, wherein step of determining a respective performance level of one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units includes one or more of interpreting the respective deep statistics to produce respective interpreted deep statistics, comparing the respective interpreted deep statistics to one or more templates of performance levels to produce respective comparisons, calculating respective performance scores based on the respective comparisons, and interpreting the respective performance scores to produce respective performance levels.

6. The method of claim 1, wherein step of determining whether to access one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units is based on whether the respective performance level of the one or more of the respective plurality of memories is above or below a threshold.

7. The method of claim 6, further including selecting another of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the another of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

8. The method of claim 7, further including generating a new dispersed storage network address for new data storage, where the new dispersed storage network address for new storage is associated with the another of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

9. A dispersed storage processing unit for selecting memory resources in a dispersed storage network, the dispersed storage network including a plurality of dispersed storage units, the plurality of dispersed storage units each including a respective plurality of memories, the dispersed storage processing unit comprising:

a communications interface;

a memory; and

a computer processor;

where the memory includes instructions for causing the computer processor to;

obtain respective deep statistics for the respective plurality of memories of one or more of the plurality of dispersed storage units;

determine a respective performance level of one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units; and

determine whether to access one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

10. The dispersed storage processing unit of claim 9, wherein the respective deep statistics pertain to respective ranges of slice names assigned to the respective plurality of memories of the one or more of the plurality of dispersed storage units.

11. The dispersed storage processing unit of claim 9, wherein the respective deep statistics include one or more of a memory device identifier, a slice name range, a throughput level, a process loading level indicator (IOPS), a number of read operations per unit of time, a number of write operations per unit of time, a number of delete operations per unit of time, an error count, an error count percentage, a number of errors per unit of time, a read latency level, a write latency level, a storage capacity utilization level, a storage capacity level, a number of requests received per unit of time, a rebuild activity rate, a rebalancing activity rate, and a migration activity rate.

12. The dispersed storage processing unit of claim 9, wherein the memory includes instructions for further causing computer the processor to issue a deep statistics request, interpret a deep statistics response, and extract the respective deep statistics from a recovered data object stored as at least one set of data object slices.

13. The dispersed storage processing unit of claim 9, wherein the memory includes instructions for further causing the computer processor to interpret the respective deep statistics to produce respective interpreted deep statistics, compare the respective interpreted deep statistics to one or more templates of performance levels to produce respective comparisons, calculate respective performance scores based on the respective comparisons, and interpret the respective performance scores to produce respective performance levels.

14. The dispersed storage processing unit of claim 9, wherein the instructions for causing the computer processor to determine whether to access one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units is based on whether the respective performance level of the one or more of the respective plurality of memories is above or below a threshold.

15. The dispersed storage processing unit of claim 14, wherein the memory includes instructions for further causing the computer processor to select another of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the another of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

16. The dispersed storage processing unit of claim 15, wherein the memory includes instructions for further causing the computer processor to generate a new dispersed storage network address for new data storage, where the new dispersed storage network address for new storage is associated with the another of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

17. A dispersed storage network comprising:

a plurality of dispersed storage units, the plurality of dispersed storage units each including a respective plurality of memories;

a dispersed storage processing units including;

a communications interface;

a memory; and

a computer processor;

where the memory includes instructions for causing the computer processor to;

obtain respective deep statistics for the respective plurality of memories of one or more of the plurality of dispersed storage units;

determine a respective performance level of one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units; and

determine whether to access one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units based on the respective performance level of the one of the one or more of the respective plurality of memories of the one or more of the plurality of dispersed storage units.

18. The dispersed storage network of claim 17, wherein the respective deep statistics pertain to respective ranges of slice names assigned to the respective plurality of memories of the one or more of the plurality of dispersed storage units.

19. The dispersed storage network of claim 17, wherein the respective deep statistics include one or more of a memory device identifier, a slice name range, a throughput level, a process loading level indicator (IOPS), a number of read operations per unit of time, a number of write operations per unit of time, a number of delete operations per unit of time, an error count, an error count percentage, a number of errors per unit of time, a read latency level, a write latency level, a storage capacity utilization level, a storage capacity level, a number of requests received per unit of time, a rebuild activity rate, a rebalancing activity rate, and a migration activity rate.

20. The dispersed storage network of claim 17, wherein the memory includes instructions for further causing computer the processor to issue a deep statistics request, interpret a deep statistics response, and extract the respective deep statistics from a recovered data object stored as at least one set of data object slices.