Extra write scaling for performance and reliability

• US 10,241,692 B2
• Filed: 08/26/2016
• Issued: 03/26/2019
• Est. Priority Date: 08/31/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg R., Resch, Jason K., Wozniak, Ethan S.
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): Dare, Ryan
• Application Number:
  US15/248,939
• Publication Number:
  US 20170060440A1
• Time in Patent Office: 942 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. For example, the computing device monitors storage unit (SU)-based write transfer rates and SU-based write failure rates associated with each of the SUs for a write request of encoded data slices (EDSs) to the SUs within the DSN. The computing device generates and maintains a SU write performance distribution based on monitoring of the SU-based write transfer rates and the SU-based write failure rates and adaptively adjusts a trimmed write threshold number of EDSs and/or a target width of EDSs for write requests of sets of EDSs to the SUs within the DSN.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

monitor, via the interface, storage unit (SU)-based write transfer rates and SU-based write failure rates associated respectively with each of a plurality of storage units (SUs) for a write request of a set of encoded data slices (EDSs) that is transmitted via the DSN to the plurality of SUs within the DSN, wherein a data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs that is of pillar width, wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

generate and maintain a SU write performance distribution based on monitoring, via the interface, of the SU-based write transfer rates and the SU-based write failure rates; and

adaptively adjust at least one of a trimmed write threshold number of EDSs or a target width of EDSs for write requests of sets of EDSs that are transmitted via the DSN to the plurality of SUs within the DSN including the write request of the set of EDSs that is transmitted via the DSN to the plurality of SUs within the DSN based on favorable or unfavorable comparison of the SU write performance distribution to an acceptable SU write performance distribution.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

adaptively adjust the trimmed write threshold number of EDSs to be below the write threshold number of EDSs and greater than or equal to the decode threshold number of EDSs when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution; and

adaptively adjust the trimmed write threshold number of EDSs to be same as the write threshold number of EDSs when the SU write performance distribution compares favorably to the acceptable SU write performance distribution.

3. The computing device of claim 2, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

adaptively adjust the trimmed write threshold number of EDSs to be a first number below the write threshold number of EDSs and greater than or equal to the decode threshold number of EDSs when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a first unfavorable comparison range; and

adaptively adjust the trimmed write threshold number of EDSs to be a second number below the write threshold number of EDSs and greater than or equal to the decode threshold number of EDSs when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a second unfavorable comparison range.

4. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

adaptively increase the target width of EDSs to be greater than the pillar width to specify a subset number of redundant EDSs to be generated from the set of EDSs to be included in at least one write request of at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution; and

adaptively adjust the target width of EDSs to be same as the pillar width when the SU write performance distribution compares favorably to the acceptable SU write performance distribution.

5. The computing device of claim 4, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

adaptively increase the target width of EDSs to be greater than the pillar width to specify a first subset number of redundant EDSs to be generated from the set of EDSs to be included in the at least one write request of the at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a first unfavorable comparison range; and

adaptively increase the target width of EDSs to be greater than the pillar width to specify a second subset number of redundant EDSs to be generated from the set of EDSs to be included in the at least one write request of the at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a second unfavorable comparison range.

6. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

adaptively increase the target width of EDSs to be greater than the pillar width to specify one or more redundant copies of the set of EDSs to be generated from the set of EDSs to be included in at least one write request of at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution; and

adaptively adjust the target width of EDSs to be same as the pillar width when the SU write performance distribution compares favorably to the acceptable SU write performance distribution.

7. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

increase a time interval between write requests of sets of EDSs to the plurality of SUs within the DSN to an extended time interval when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution over a first period of time; and

reduce the time interval between write requests of sets of EDSs to the plurality of SUs within the DSN from the extended time interval to another extended time interval or the time interval when the SU write performance distribution compares favorably to the acceptable SU write performance distribution over a second period of time following the first period of time.

8. The computing device of claim 1, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

9. The computing device of claim 1 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

10. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

11. A method for execution by a computing device, the method comprising:

monitoring, via an interface of the computing device that is configured to interface and communicate with a dispersed storage network (DSN), storage unit (SU)-based write transfer rates and SU-based write failure rates associated respectively with each of a plurality of storage units (SUs) for a write request of a set of encoded data slices (EDSs) that is transmitted via the DSN to the plurality of SUs within the DSN via the interface of the computing device implemented to interface and communicate with the DSN, wherein a data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs that is of pillar width, wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

generating and maintaining a SU write performance distribution based on monitoring, via the interface, of the SU-based write transfer rates and the SU-based write failure rates; and

adaptively adjusting at least one of a trimmed write threshold number of EDSs or a target width of EDSs for write requests of sets of EDSs that are transmitted via the DSN to the plurality of SUs within the DSN including the write request of the set of EDSs that is transmitted via the DSN to the plurality of SUs within the DSN based on favorable or unfavorable comparison of the SU write performance distribution to an acceptable SU write performance distribution.

12. The method of claim 11 further comprising:

adaptively adjusting the trimmed write threshold number of EDSs to be below the write threshold number of EDSs and greater than or equal to the decode threshold number of EDSs when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution; and

adaptively adjusting the trimmed write threshold number of EDSs to be same as the write threshold number of EDSs when the SU write performance distribution compares favorably to the acceptable SU write performance distribution.

13. The method of claim 12 further comprising:

adaptively adjusting the trimmed write threshold number of EDSs to be a first number below the write threshold number of EDSs and greater than or equal to the decode threshold number of EDSs when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a first unfavorable comparison range; and

adaptively adjusting the trimmed write threshold number of EDSs to be a second number below the write threshold number of EDSs and greater than or equal to the decode threshold number of EDSs when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a second unfavorable comparison range.

14. The method of claim 11 further comprising:

adaptively increasing the target width of EDSs to be greater than the pillar width to specify a subset number of redundant EDSs to be generated from the set of EDSs to be included in at least one write request of at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution; and

adaptively adjusting the target width of EDSs to be same as the pillar width when the SU write performance distribution compares favorably to the acceptable SU write performance distribution.

15. The method of claim 14 further comprising:

adaptively increasing the target width of EDSs to be greater than the pillar width to specify a first subset number of redundant EDSs to be generated from the set of EDSs to be included in the at least one write request of the at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a first unfavorable comparison range; and

adaptively increasing the target width of EDSs to be greater than the pillar width to specify a second subset number of redundant EDSs to be generated from the set of EDSs to be included in the at least one write request of the at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution within a second unfavorable comparison range.

16. The method of claim 11 further comprising:

adaptively increasing the target width of EDSs to be greater than the pillar width to specify one or more redundant copies of the set of EDSs to be generated from the set of EDSs to be included in at least one write request of at least one set of EDSs to the plurality of SUs within the DSN when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution; and

adaptively adjusting the target width of EDSs to be same as the pillar width when the SU write performance distribution compares favorably to the acceptable SU write performance distribution.

17. The method of claim 11 further comprising:

increasing a time interval between write requests of sets of EDSs to the plurality of SUs within the DSN to an extended time interval when the SU write performance distribution compares unfavorably to the acceptable SU write performance distribution over a first period of time; and

reducing the time interval between write requests of sets of EDSs to the plurality of SUs within the DSN from the extended time interval to another extended time interval or the time interval when the SU write performance distribution compares favorably to the acceptable SU write performance distribution over a second period of time following the first period of time.

18. The method of claim 11, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

19. The method of claim 11, wherein the computing device is a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

20. The method of claim 11, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).