Data access optimization protocol in a dispersed storage network

• US 10,289,342 B2
• Filed: 12/15/2017
• Issued: 05/14/2019
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Dhuse, Greg R., Khadiwala, Ravi V., Leggette, Wesley B.
• Primary Examiner(s): Nguyen, Hiep T
• Application Number:
  US15/843,637
• Publication Number:
  US 20180107428A1
• Time in Patent Office: 515 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method includes detecting triggering of establishing a data access optimization protocol for at least a portion of a dispersed storage network (DSN). The establishing the data access optimization protocol includes determining an error rate parameter based on subscription data regarding the at least a portion of the DSN, determining system error rate of the at least a portion of the DSN, and establishing a data access threshold plus protocol based on the error rate parameter and the system error rate. The data access threshold plus protocol includes a value greater than a threshold number. When the data access optimization protocol is established, the method further includes generating a set of data access requests for the set of encoded data slices in accordance with the data access optimization protocol, and sending the set of data access requests to a set of storage units affiliated with the at least a portion of the DSN.

Claims

1. A method comprises

detecting, by a computing device of a dispersed storage network (DSN), a triggering of establishing a data access optimization protocol for at least a portion of the DSN, wherein establishing the data access optimization protocol includes:

determining, by the computing device, an error rate parameter based on subscription data regarding the at least a portion of the DSN;

determining, by the computing device, system error rate of the at least a portion of the DSN;

establishing, by the computing device, a data access threshold plus protocol based on the error rate parameter and the system error rate, wherein the data access threshold plus protocol corresponds to at least a portion of the data access optimization protocol, wherein, for a set of encoded data slices of a data segment, a threshold number of encoded data slices is required to execute a data access request, and wherein the data access threshold plus protocol includes a value that is greater than the threshold number; and

when the data access optimization protocol has been established;

generating, by the computing device, a set of data access requests for the set of encoded data slices in accordance with the data access optimization protocol; and

sending, by the computing device, the set of data access requests to a set of storage units affiliated with the at least a portion of the DSN.

2. The method of claim 1, wherein the triggering of establishing the data access optimization protocol comprises one or more of:

receiving, by the computing device, the data access request regarding the data segment;

receiving, by the computing device, a command to establish the data access optimization protocol on a periodic basis;

determining, by the computing device, identity of a user issuing the data access request for the data segment;

determining, by the computing device, that the system error rate is below a threshold;

determining, by the computing device, that the error rate parameter exceeds an upper threshold; and

receiving, by the computing device, a system command to establish the data access optimization protocol.

3. The method of claim 1, wherein the establishing the data access threshold plus protocol comprises:

performing, by the computing device, a function on the error rate parameter and the system error rate to determine an offset value; and

establishing, by the computing device, the data access threshold plus protocol based on the threshold number and the offset value.

4. The method of claim 3, wherein the generating the set of data access requests comprises:

generating, by the computing device, a threshold number plus offset value number of data access requests for the threshold number plus offset value number of encoded data slices, wherein the threshold number plus offset value number is less than a pillar width number of encoded data slices.

5. The method of claim 3, wherein the establishing the data access threshold plus protocol comprises:

determining, by the computing device, an initial value for the threshold number plus the offset value;

estimating, by the computing device, processing resource consumption of the at least a portion of the DSN based on the initial value;

estimating, by the computing device, processing resource consumption of the at least a portion of the DSN based on pillar width data access;

when the estimated processing resource consumption based on the pillar width data access is within a processing resource consumption threshold of the estimated processing resource consumption based on the initial value, utilizing, by the computing device, the pillar width data access as the data access threshold plus protocol; and

when the estimated processing resource consumption based on the pillar width data access is over the processing resource consumption threshold, utilizing, by the computing device, the initial value as the data access threshold plus protocol.

6. The method of claim 1 further comprises:

receiving, by the computing device, a read request as the data access request;

establishing, by the computing device, the data access threshold plus protocol by;

performing, by the computing device, a read based function on the error rate parameter and the system error rate to determine an offset value; and

establishing, by the computing device, the data access threshold plus protocol based on the threshold number and the offset value, wherein the threshold number is a decode threshold number of error encoded data slices required to reconstruct the data segment.

7. The method of claim 1 further comprises:

receiving, by the computing device, a write request as the data access request;

establishing, by the computing device, the data access threshold plus protocol by;

performing, by the computing device, a write based function on the error rate parameter and the system error rate to determine an offset value; and

establishing, by the computing device, the data access threshold plus protocol based on the threshold and the offset value, wherein the threshold is a write threshold number of error encoded data slices.

8. A computing device of a dispersed storage network (DSN), the computing device comprises:

an interface;

memory; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

detect a triggering of establishing a data access optimization protocol for at least a portion of the DSN, wherein establishing the data access optimization protocol includes;

determining an error rate parameter based on subscription data regarding the at least a portion of the DSN;

determining system error rate of the at least a portion of the DSN;

establishing a data access threshold plus protocol based on the error rate parameter and the system error rate, wherein the data access threshold plus protocol corresponds to at least a portion of the data access optimization protocol, wherein, for a set of encoded data slices of a data segment, a threshold number of encoded data slices is required to execute a data access request, and wherein the data access threshold plus protocol includes a value that is greater than the threshold number; and

when the data access optimization protocol has been established;

generate a set of data access requests for the set of encoded data slices in accordance with the data access optimization protocol; and

send the set of data access requests to a set of storage units affiliated with the at least a portion of the DSN.

9. The computing device of claim 8, wherein the processing module is operable to detect the triggering of establishing the data access optimization protocol by one or more of:

receiving the data access request regarding the data segment;

receiving a command to establish the data access optimization protocol on a periodic basis;

determining identity of a user issuing the data access request for the data segment;

determining that the system error rate is below a threshold;

determining that the error rate parameter exceeds an upper threshold; and

receiving a system command to establish the data access optimization protocol.

10. The computing device of claim 8, wherein the processing module is operable to establish the data access threshold plus protocol by:

performing a function on the error rate parameter and the system error rate to determine an offset value; and

establishing the data access threshold plus protocol based on the threshold number and the offset value.

11. The computing device of claim 10, wherein the processing module is operable to generate the set of data access requests by:

generating a threshold number plus offset value number of data access requests for the threshold number plus offset value number of encoded data slices, wherein the threshold number plus offset value number is less than a pillar width number of encoded data slices.

12. The computing device of claim 10, wherein the processing module is operable to establish the data access threshold plus protocol by:

determining an initial value for the threshold number plus the offset value;

estimating processing resource consumption of the at least a portion of the DSN based on the initial value;

estimating processing resource consumption of the at least a portion of the DSN based on pillar width data access;

when the estimated processing resource consumption based on the pillar width data access is within a processing resource consumption threshold of the estimated processing resource consumption based on the initial value, utilizing the pillar width data access as the data access threshold plus protocol; and

when the estimated processing resource consumption based on the pillar width data access is over the processing resource consumption threshold, utilizing the initial value as the data access threshold plus protocol.

13. The computing device of claim 8, wherein the processing module is further operable to:

receive a read request as the data access request;

establish the data access threshold plus protocol by;

performing a read based function on the error rate parameter and the system error rate to determine an offset value; and

establishing the data access threshold plus protocol based on the threshold number and the offset value, wherein the threshold number is a decode threshold number of error encoded data slices required to reconstruct the data segment.

14. The computing device of claim 8, wherein the processing module is further operable to:

receive a write request as the data access request;

establish the data access threshold plus protocol by;

performing a write based function on the error rate parameter and the system error rate to determine an offset value; and

establishing the data access threshold plus protocol based on the threshold and the offset value, wherein the threshold is a write threshold number of error encoded data slices.