Throttled real-time writes

• US 10,360,106 B2
• Filed: 01/18/2017
• Issued: 07/23/2019
• Est. Priority Date: 12/12/2011
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg R., Leggette, Wesley B., Resch, Jason K.
• Primary Examiner(s): Blair, April Y
• Assistant Examiner(s): Gandhi, Dipakkumar B
• Application Number:
  US15/409,386
• Publication Number:
  US 20170123920A1
• Time in Patent Office: 916 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 2463/061 applying further key derivation, e.g. deriving traffic keys from a pair-wise master key
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable based on the operational instructions, is configured to perform various operations. The computing device determines data access rates corresponding respectively to storage units (SUs). In certain situations, the computing device selects at least a data access threshold number of SUs excluding a first slowest SU having the first slowest data access rate to service data access request(s) for set(s) of encoded data slices (EDSs) corresponding to a data object. The computing device facilitates servicing of the data access request(s) for the set(s) of EDSs by the at least a data access threshold number of SUs that excludes the first slowest SU having the first slowest data access rate.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

determine a plurality of data access rates corresponding respectively to a plurality of storage units (SUs) within the DSN;

when a difference between a first slowest data access rate and a second slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to a performance parameter as being greater than or equal to the performance parameter, select at least a data access threshold number of SUs among the plurality of SUs that excludes a first slowest SU having the first slowest data access rate to service at least one data access request for at least one set of encoded data slices (EDSs) corresponding to a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs), wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN, and wherein the data access threshold number corresponds to at least one of the decode threshold number, the read threshold number, or the write threshold number; and

facilitate servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

dispersed error encode (DSE) the data object to generate sets of EDSs that include the set of EDSs;

transmit a plurality of write requests to the plurality of SUs within the DSN;

determine a plurality of data ingest rates corresponding respectively to the plurality of SUs within the DSN based on the plurality of write requests;

when another difference between a first slowest data ingest rate and a second slowest data ingest rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to a data ingest rate performance parameter as being greater than or equal to the data ingest rate performance parameter, select at least a write threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data ingest rate to service at least one write request for the at least one set of EDSs corresponding to the data object;

determine a transmit data rate that compares favorably to a slowest data ingest rate as being no faster than the slowest data ingest rate among the write threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data ingest rate; and

transmit at least a write threshold number of write requests to the at least a write threshold number of SUs based on the transmit data rate to facilitate servicing of the at least one write request for the at least one set of EDSs corresponding to the data object.

3. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

transmit a plurality of read requests to the plurality of SUs within the DSN;

determine a plurality of data read rates corresponding respectively to the plurality of SUs within the DSN based on the plurality of read requests;

when another difference between a first slowest data read rate and a second slowest data read rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to a data read rate performance parameter as being greater than or equal to the data read rate performance parameter, select at least a read threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data read rate to service at least read request for the at least one set of EDSs corresponding to the data object;

determine a transmit data rate that compares favorably to a slowest data read rate as being no faster than the slowest data read rate among the at least a read threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data read rate; and

transmit at least a read threshold number of read requests to the at least a read threshold number of SUs based on the transmit data rate to facilitate servicing of the at least read request for the at least one set of EDSs corresponding to the data object.

4. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

when the difference between the first slowest data access rate and the second slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares unfavorably to the performance parameter as being less than the performance parameter, and when another difference between the second slowest data access rate and a third slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to the performance parameter as being greater than or equal to the performance parameter, select the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate and a second slowest SU having the second slowest data access rate to service at least one data access request for the at least one set of EDSs corresponding to the data object; and

facilitate servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate and the second slowest SU having the second slowest data access rate.

5. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

facilitate rebuilding of at least one remaining set of EDSs excluded from the servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate.

6. The computing device of claim 1, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

7. The computing device of claim 1 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

determine a plurality of data access rates corresponding respectively to a plurality of storage units (SUs) within the DSN;

when a first difference between a first slowest data access rate and a second slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares unfavorably to a performance parameter as being less than the performance parameter and when a second difference between the first slowest data access rate and a third slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to the performance parameter as being greater than or equal to the performance parameter, select at least a data access threshold number of SUs among the plurality of SUs that excludes a first slowest SU having the first slowest data access rate and a second slowest SU having the second slowest data access rate to service at least one data access request for at least one set of encoded data slices (EDSs) corresponding to a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs), wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN, and wherein the data access threshold number corresponds to at least one of the decode threshold number, the read threshold number, or the write threshold number;

facilitate servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate and the second slowest SU having the second slowest data access rate; and

facilitate rebuilding of at least one remaining set of EDSs excluded from the servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate.

10. The computing device of claim 9, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

dispersed error encode (DSE) the data object to generate sets of EDSs that include the set of EDSs;

transmit a plurality of write requests to the plurality of SUs within the DSN;

determine a plurality of data ingest rates corresponding respectively to the plurality of SUs within the DSN based on the plurality of write requests;

when a third difference between a first slowest data ingest rate and a second slowest data ingest rate among the plurality of data ingest rates corresponding respectively to the plurality of SUs within the DSN compares unfavorably to a data ingest rate performance parameter as being less than the data ingest rate performance parameter and when a fourth difference between the first slowest data ingest rate and a third slowest data ingest rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to the data ingest rate performance parameter as being greater than or equal to the data ingest rate performance parameter, select at least a write threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data ingest rate and a second slowest SU having the second slowest data ingest rate to service at least one write request for the at least one set of EDSs corresponding to the data object;

determine a transmit data rate that compares favorably to a slowest data ingest rate as being no faster than the slowest data ingest rate among the at least a write threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data ingest rate; and

transmit at least a write threshold number of write requests to the at least a write threshold number of SUs based on the transmit data rate to facilitate servicing of the at least one write request for the at least one set of EDSs corresponding to the data object.

11. The computing device of claim 9, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

transmit a plurality of read requests to the plurality of SUs within the DSN;

determine a plurality of data read rates corresponding respectively to the plurality of SUs within the DSN based on the plurality of read requests;

when a third difference between a first slowest data read rate and a second slowest data read rate among the plurality of data read rate corresponding respectively to the plurality of SUs within the DSN compares unfavorably to a data read rate performance parameter as being less than the data read rate performance parameter, and when a fourth difference between the first slowest data read rate and a third slowest data read rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to the data read rate performance parameter as being greater than or equal to the data read rate performance parameter, select at least a read threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data read rate and a second slowest SU having the second slowest data read rate to service at least one read request for the at least one set of EDSs corresponding to the data object;

determine a transmit data rate that compares favorably to a slowest data read rate as being no faster than the slowest data read rate among the at least a read threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data read rate; and

transmit at least a read threshold number of read requests to the at least a read threshold number of SUs based on the transmit data rate to facilitate servicing of the at least read request for the at least one set of EDSs corresponding to the data object.

12. The computing device of claim 9 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

determining a plurality of data access rates corresponding respectively to a plurality of storage units (SUs) within a dispersed or distributed storage network (DSN);

when a difference between a first slowest data access rate and a second slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to a performance parameter as being greater than or equal to the performance parameter, selecting at least a data access threshold number of SUs among the plurality of SUs that excludes a first slowest SU having the first slowest data access rate to service at least one data access request for at least one set of encoded data slices (EDSs) corresponding to a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs), wherein a decode threshold number of EDSs are needed to recover the data segment, wherein a read threshold number of EDSs provides for reconstruction of the data segment, wherein a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN, and wherein the data access threshold number corresponds to at least one of the decode threshold number, the read threshold number, or the write threshold number; and

via an interface of the computing device that is configured to interface and communicate with the DSN, facilitating servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate.

15. The method of claim 14 further comprising:

dispersed error encoding the data object to generate sets of EDSs that include the set of EDSs;

transmitting, via the interface, a plurality of write requests to the plurality of SUs within the DSN;

determining a plurality of data ingest rates corresponding respectively to the plurality of SUs within the DSN based on the plurality of write requests;

when another difference between a first slowest data ingest rate and a second slowest data ingest rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to a data ingest rate performance parameter as being greater than or equal to the data ingest rate performance parameter, selecting at least a write threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data ingest rate to service at least one write request for the at least one set of EDSs corresponding to the data object;

determining a transmit data rate that compares favorably to a slowest data ingest rate as being no faster than the slowest data ingest rate among the at least a write threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data ingest rate; and

transmitting, via the interface, at least a write threshold number of write requests to the at least a write threshold number of SUs based on the transmit data rate to facilitate servicing of the at least one write request for the at least one set of EDSs corresponding to the data object.

16. The method of claim 14 further comprising:

transmitting, via the interface, a plurality of read requests to the plurality of SUs within the DSN;

determining a plurality of data read rates corresponding respectively to the plurality of SUs within the DSN based on the plurality of read requests;

when another difference between a first slowest data read rate and a second slowest data read rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to a data read rate performance parameter as being greater than or equal to the data read rate performance parameter, selecting at least a read threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data read rate to service at least read request for the at least one set of EDSs corresponding to the data object;

determining a transmit data rate that compares favorably to a slowest data read rate as being no faster than the slowest data read rate among the at least a read threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data read rate; and

transmitting, via the interface, at least a read threshold number of read requests to the at least a read threshold number of SUs based on the transmit data rate to facilitate servicing of the at least read request for the at least one set of EDSs corresponding to the data object.

17. The method of claim 14 further comprising:

when the difference between the first slowest data access rate and the second slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares unfavorably to the performance parameter as being less than the performance parameter, and when another difference between the second slowest data access rate and a third slowest data access rate among the plurality of data access rates corresponding respectively to the plurality of SUs within the DSN compares favorably to the performance parameter as being greater than or equal to the performance parameter, selecting the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate and a second slowest SU having the second slowest data access rate to service at least one data access request for the at least one set of EDSs corresponding to the data object; and

facilitating servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate and the second slowest SU having the second slowest data access rate.

18. The method of claim 14 further comprising:

via the interface of the computing device, facilitating rebuilding of at least one remaining set of EDSs excluded from the servicing of the at least one data access request for the at least one set of EDSs corresponding to the data object by the at least a data access threshold number of SUs among the plurality of SUs that excludes the first slowest SU having the first slowest data access rate.

19. The method of claim 14, wherein the computing device includes a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).