Modifying information dispersal algorithm configurations in a dispersed storage network

• US 10,057,351 B2
• Filed: 01/06/2017
• Issued: 08/21/2018
• Est. Priority Date: 09/13/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Kazi, Asimuddin
• Primary Examiner(s): Giardino, Jr., Mark A
• Application Number:
  US15/400,092
• Publication Number:
  US 20170118288A1
• Time in Patent Office: 592 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

A method for execution by a dispersed storage (DS) client module includes receiving a write request for a first data object. A set of storage units associated with the first data object are identified, and an availability level is determined. The DS client module determines to modify dispersal parameters associated with the set based on the availability level, and modified dispersal parameters are determined based on current dispersal parameters and the availability level. Encoded slices are generated by performing an encoding function on the first data object using the modified dispersal parameters, and the slices are sent to the storage units. A second data object stored in the identified set of storage units is recovered by utilizing the current dispersal parameters. Encoded slices are generated by performing an encoding function on the second data object using the modified dispersal parameters, and the slices are sent to the storage units.

Claims

1. A method for execution by a distributed storage (DS) client module that includes a processor, the method comprises:

receiving a write request to store a first data object;

identifying a set of storage units associated with the first data object;

determining an availability level of the identified set of storage units;

determining to modify dispersal parameters associated with the set of storage units based on the availability level of the set of storage units;

determining modified dispersal parameters based on current dispersal parameters and the availability level of the set of storage units;

generating a first plurality of sets of encoded data slices by performing an encoding function on the first data object using the modified dispersal parameters;

sending the plurality of sets of encoded data slices to the identified set of storage units;

recovering a second data object stored in the identified set of storage units by utilizing the current dispersal parameters;

generating a second plurality of sets of encoded data slices by performing an encoding function on the second data object using the modified dispersal parameters; and

sending the second plurality of sets of encoded data slices to the identified set of storage units.

2. The method of claim 1, wherein identifying the set of storage units includes identifying a vault based on an identifier of a requesting entity by a registry lookup, wherein the set of storage units is identified based on identifier of the vault based on a registry lookup.

3. The method of claim 1, wherein the availability level is based on at least one of:

a number of storage units of the identified set of storage units that are operational.

4. The method of claim 1, wherein determining the availability level is based on at least one of:

initiating a test, initiating a query, receiving a response, or performing a lookup.

5. The method of claim 1, wherein the DS client module determines to modify dispersal parameters in response to determining that the availability level of the set of storage units compares unfavorably to an availability level threshold.

6. The method of claim 1, wherein determining the modified dispersal parameters includes at least one of:

changing a pillar width of the current dispersal parameters or changing a decode threshold of the current dispersal parameters.

7. The method of claim 1, wherein determining the modified dispersal parameters includes calculating an estimated reliability level corresponding to the modified dispersal parameters and determining that the estimated reliability level compares favorably to a reliability level threshold.

8. The method of claim 1, wherein determining the modified dispersal parameters includes increasing a pillar width of the current dispersal parameters in response to determining that the availability level has increased from a previous availability level.

9. The method of claim 1, wherein determining the modified dispersal parameters includes decreasing a pillar width of the current dispersal parameters in response to determining that the availability level has decreased from a previous availability level.

10. The method of claim 9, wherein the pillar width is decreased in response to determining that at least one storage unit of the set of storage units has become unavailable.

11. A processing system of a dispersed storage (DS) client module comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

receive a write request to store a first data object;

identify a set of storage units associated with the first data object;

determine an availability level of the identified set of storage units;

determine to modify dispersal parameters associated with the set of storage units based on the availability level of the set of storage units;

determine modified dispersal parameters based on current dispersal parameters and the availability level of the set of storage units;

generate a first plurality of sets of encoded data slices by performing an encoding function on the first data object using the modified dispersal parameters;

send the plurality of sets of encoded data slices to the identified set of storage units;

recover a second data object stored in the identified set of storage units by utilizing the current dispersal parameters;

generate a second plurality of sets of encoded data slices by performing an encoding function on the second data object using the modified dispersal parameters; and

send the second plurality of sets of encoded data slices to the identified set of storage units.

12. The processing system of claim 11, wherein identifying the set of storage units includes identifying a vault based on an identifier of a requesting entity by a registry lookup, wherein the set of storage units is identified based on identifier of the vault based on a registry lookup.

13. The processing system of claim 11, wherein the availability level is based on at least one of:

a number of storage units of the identified set of storage units that are operational.

14. The processing system of claim 11, wherein the DS client module determines to modify dispersal parameters in response to determining that the availability level of the set of storage units compares unfavorably to an availability level threshold.

15. The processing system of claim 11, wherein determining the modified dispersal parameters includes at least one of:

changing a pillar width of the current dispersal parameters or changing a decode threshold of the current dispersal parameters.

16. The processing system of claim 11, wherein determining the modified dispersal parameters includes calculating an estimated reliability level corresponding to the modified dispersal parameters and determining that the estimated reliability level compares favorably to a reliability level threshold.

17. The processing system of claim 11, wherein determining the modified dispersal parameters includes increasing a pillar width of the current dispersal parameters in response to determining that the availability level has increased from a previous availability level.

18. The processing system of claim 11, wherein determining the modified dispersal parameters includes decreasing a pillar width of the current dispersal parameters in response to determining that the availability level has decreased from a previous availability level.

19. The processing system of claim 18, wherein the pillar width is decreased in response to determining that at least one storage unit of the set of storage units has become unavailable.

20. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

receive a write request to store a first data object;

identify a set of storage units associated with the first data object;

determine an availability level of the identified set of storage units;

determine to modify dispersal parameters associated with the set of storage units based on the availability level of the set of storage units;

determine modified dispersal parameters based on current dispersal parameters and the availability level of the set of storage units;

generate a first plurality of sets of encoded data slices by performing an encoding function on the first data object using the modified dispersal parameters;

send the plurality of sets of encoded data slices to the identified set of storage units;

recover a second data object stored in the identified set of storage units by utilizing the current dispersal parameters;

generate a second plurality of sets of encoded data slices by performing an encoding function on the second data object using the modified dispersal parameters; and

send the second plurality of sets of encoded data slices to the identified set of storage units.