Handling storage unit latency in a dispersed storage network

• US 9,996,283 B2
• Filed: 07/29/2016
• Issued: 06/12/2018
• Est. Priority Date: 08/31/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K.
• Primary Examiner(s): Bertram, Ryan
• Application Number:
  US15/224,118
• Publication Number:
  US 20170060454A1
• Time in Patent Office: 683 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a dispersed storage and task (DST) processing unit includes generating storage unit performance data based on a performance threshold value and storage unit performance values of storage units in a storage unit write set. Storage unit write set data indicating a new storage unit write set based on the storage unit performance data is generated, where at least one slow-performing storage unit is removed from the storage unit write set to create the new storage unit write set when the at least one slow-performing storage unit has a storage unit performance value that compares unfavorably to the performance threshold value. A plurality of write requests are generated for transmission to the new storage unit write set via a network, each including a data slices to be written to a corresponding storage unit of the new storage unit write set.

Claims

1. A method for execution by a dispersed storage and task (DST) processing unit that includes a processor, the method comprises:

generating first storage unit performance data based on a first performance threshold value and a plurality of storage unit performance values corresponding to an original plurality of storage units of an original storage unit write set;

generating storage unit write set data indicating a new storage unit write set based on the first storage unit performance data, wherein at least one first slow-performing storage unit is removed from the original storage unit write set to create the new storage unit write set when the first storage unit performance data indicates that the at least one first slow-performing storage unit has an at least one corresponding storage unit performance value that compares unfavorably to the first performance threshold value;

generating a plurality of write requests for transmission to each of the storage units of the new storage unit write set via a network, wherein each write request includes one of a plurality of data slices to be written to a corresponding storage unit of the new storage unit write set; and

generating second storage unit performance data when the at least one first slow-performing storage unit has a corresponding storage unit performance value that compares unfavorably to the first performance threshold value, wherein the second storage unit performance data is based on a second performance threshold value that compares favorably to the first performance threshold value, and further based on the plurality of storage unit performance values;

wherein generating the storage unit write set data further includes removing at least one second slow-performing storage unit from the original storage unit write set to create the new storage unit write set when the second storage unit performance data indicates that the at least one second slow-performing storage unit has an at least one corresponding storage unit value that compares unfavorably to the second performance threshold value.

2. The method of claim 1, wherein the at least one corresponding storage unit value the second slow-performing storage unit compares unfavorably to the second performance threshold value and compares favorably to the first performance threshold value.

3. The method of claim 1, wherein generating the storage unit write set data includes creating the new storage unit write set by including all of the storage units of the original storage unit write set when the first storage unit performance data indicates that all of the plurality of storage unit performance values compare favorably to the first threshold performance value.

4. The method of claim 1, further comprising generating the plurality of data slices by performing an encoding function on a data object to be written to storage, wherein a parameter of the encoding function is the size of the new storage unit write set, and wherein the number of data slices generated is equal to the size of the new storage unit write set.

5. The method of claim 1, further comprising generating an original plurality of data slices by performing an encoding function on a data object to be written to storage, wherein the number of data slices generated is equal to the size of the original storage unit write set, wherein a subset of the original plurality of data slices to be written to storage is selected based on the new storage unit write set, and wherein the plurality of data slices to be written to corresponding storage units of the new storage unit write set corresponds to the data slices of the subset.

6. The method of claim 1, further comprising receiving the plurality of storage unit performance values from the original plurality of storage units of an original storage unit write set via the network.

7. The method of claim 6, further comprising generating a plurality of performance data requests, each for transmission to one of the original plurality of storage units of the original storage unit write set via the network, and wherein the plurality of storage unit performance values are received in response to the plurality of performance data requests.

8. The method of claim 1, wherein generating the storage unit write set data is further based on a minimum write threshold, and wherein the size of the new storage unit write set is greater than or equal to the minimum write threshold.

9. The method of claim 8, wherein the first storage unit performance data indicates a plurality of first slow-performing storage units with corresponding storage unit performance values that compare unfavorably to the first performance threshold value, wherein a subset of the plurality of first slow-performing storage units is selected to remain in the new storage unit write set, wherein only the storage units of the plurality of first slow-performing storage units that are not included in the subset are removed from the original storage unit write set, wherein the subset is selected based on the minimum write threshold, and wherein the size of the new storage unit write set is equal to the minimum write threshold.

10. The method of claim 9, wherein selecting the subset is further based on a ranking of storage unit performance values corresponding to the first plurality of slow-performing storage units.

11. The method of claim 1, further comprising selecting the first performance threshold value based on at least one of:

current system requirements or a record of past accesses.

12. The method of claim 1, wherein the plurality of data slices correspond to a data object, further comprising selecting the first performance threshold value based on a write urgency indicator associated with the data object.

13. A processing system of a dispersed storage and task (DST) processing unit comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

generate first storage unit performance data based on a first performance threshold value and a plurality of storage unit performance values corresponding to an original plurality of storage units of an original storage unit write set;

generate storage unit write set data indicating a new storage unit write set based on the first storage unit performance data, wherein at least one first slow-performing storage unit is removed from the original storage unit write set to create the new storage unit write set when the first storage unit performance data indicates that the at least one first slow-performing storage unit has an at least one corresponding storage unit performance value that compares unfavorably to the first performance threshold value;

generate a plurality of write requests for transmission to each of the storage units of the new storage unit write set via a network, wherein each write request includes one of a plurality of data slices to be written to a corresponding storage unit of the new storage unit write set; and

generate second storage unit performance data when the at least one first slow-performing storage unit has a corresponding storage unit performance value that compare unfavorably to the first performance threshold value, wherein the second storage unit performance data is based on a second performance threshold value that compares favorably to the first performance threshold value, and further based on the plurality of storage unit performance values;

wherein generating the storage unit write set data further includes removing at least one second slow-performing storage unit from the original storage unit write set to create the new storage unit write set when the second storage unit performance data indicates that the at least one second slow-performing storage unit has an at least one corresponding storage unit value that compares unfavorably to the second performance threshold value.

14. The processing system of claim 13, wherein the at least one corresponding storage unit value the second slow-performing storage unit compares unfavorably to the second performance threshold value and compares favorably to the first performance threshold value.

15. The processing system of claim 13, wherein generating the storage unit write set data includes creating the new storage unit write set by including all of the storage units of the original storage unit write set when the first storage unit performance data indicates that all of the plurality of storage unit performance values compare favorably to the first threshold performance value.

16. The processing system of claim 13, wherein generating the storage unit write set data is further based on a minimum write threshold, and wherein the size of the new storage unit write set is greater than or equal to the minimum write threshold.

17. The processing system of claim 16, wherein the first storage unit performance data indicates a plurality of first slow-performing storage units with corresponding storage unit performance values that compare unfavorably to the first performance threshold value, wherein a subset of the plurality of first slow-performing storage units is selected to remain in the new storage unit write set, wherein only the storage units of the plurality of first slow-performing storage units that are not included in the subset are removed from the original storage unit write set, wherein the subset is selected based on the minimum write threshold, and wherein the size of the new storage unit write set is equal to the minimum write threshold.

18. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

generate first storage unit performance data based on a first performance threshold value and a plurality of storage unit performance values corresponding to an original plurality of storage units of an original storage unit write set;

generate storage unit write set data indicating a new storage unit write set based on the first storage unit performance data, wherein at least one first slow-performing storage unit is removed from the original storage unit write set to create the new storage unit write set when the first storage unit performance data indicates that the at least one first slow-performing storage unit has an at least one corresponding storage unit performance value that compares unfavorably to the first performance threshold value;

generate a plurality of write requests for transmission to each of the storage units of the new storage unit write set via a network, wherein each write request includes one of a plurality of data slices to be written to a corresponding storage unit of the new storage unit write set; and

generate second storage unit performance data when the at least one first slow-performing storage unit has a corresponding storage unit performance value that compare unfavorably to the first performance threshold value, wherein the second storage unit performance data is based on a second performance threshold value that compares favorably to the first performance threshold value, and further based on the plurality of storage unit performance values;

wherein generating the storage unit write set data further includes removing at least one second slow-performing storage unit from the original storage unit write set to create the new storage unit write set when the second storage unit performance data indicates that the at least one second slow-performing storage unit has an at least one corresponding storage unit value that compares unfavorably to the second performance threshold value.