Expanding storage capacity of a set of storage units in a distributed storage network

• US 10,079,887 B2
• Filed: 01/26/2016
• Issued: 09/18/2018
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Motwani, Manish, Wozniak, Ethan S.
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US15/006,735
• Publication Number:
  US 20160294949A1
• Time in Patent Office: 966 Days
• Field of Search: 709215
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module determining that storage of a set of storage units has been expanded and determining, for a plurality of sets of a desired number of encoded data slices, updated storage properties. For each set of the desired number of encoded data slices of at least some of the plurality of sets of the desired number of slices, the method continues with the processing module performing a scoring function using the updated storage properties to produce a per set storage resultant, identifying a sub set of storage units of the set of storage units for storing the set of the desired number of encoded data slices based on the per set storage resultant, and storing the set of the desired number of encoded data slices in the sub set of storage units.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

determining that storage of a set of storage units has been expanded;

determining, for a plurality of sets of a desired number of encoded data slices, updated storage properties based on the storage expansion of the set of storage units, wherein a number of storage units in the set of storage units exceeds a number of encoded data slices of a set of the desired number of encoded data slices of the plurality of sets of the desired number of encoded data slices; and

for each set of the desired number of encoded data slices of at least some of the plurality of sets of the desired number of encoded data slices;

performing a scoring function using the updated storage properties to produce a per set storage resultant;

identifying a sub set of storage units of the set of storage units for storing the set of the desired number of encoded data slices based on the per set storage resultant; and

storing the set of the desired number of encoded data slices in the sub set of storage units.

2. The method of claim 1, wherein the performing the scoring function further comprises:

computing a storage value for each storage unit of the set of storage units based on the updated storage properties to produce a set of storage values; and

ordering the set of storage values in accordance with the scoring function to produce an ordered set of storage values, wherein the per set storage resultant includes the ordered set of storage values.

3. The method of claim 2, wherein the identifying the sub set of storage units comprises:

selecting a grouping of the ordered set of storage values in accordance with the scoring function; and

corresponding the sub set of storage units to the storage units having a storage value in the grouping of the ordered set of storage values.

4. The method of claim 1, wherein the set of the desired number of encoded data slices comprises:

for an encoded data slice of the set of the desired number of encoded data slices;

determining whether storage location within the set of storage units has changed;

when the storage location within the set of storage units has changed;

storing the encoded data slice at a new storage location within the sub set of storage units; and

deleting the encoded data slices from an old storage location with the set of storage units.

5. The method of claim 4 further comprises:

when the storage location within the set of storage units has not changed, maintaining storage of the encoded data slices at the storage location.

6. The method of claim 1, wherein the set of the desired number of encoded data slices comprises one of:

a set of a write threshold number of encoded data slices; and

a set of an information dispersal algorithm (IDA) width number of encoded data slices, wherein a decode threshold number of encoded data slices of the desired number of encoded data slices is required to rebuild a corresponding data segment of data.

7. The method of claim 1, wherein the determining that storage of a set of storage units has been expanded comprises:

determining that the number of storage units in the set of storage units has increased in value from a previous number of storage units in the set of storage units.

8. The method of claim 1, wherein the determining that storage of a set of storage units has been expanded comprises:

determining that one or more memory devices have been added to at least one of the storage units of the set of storage units.

9. The method of claim 1, wherein the updated storage properties comprises:

information regarding the set of the desired number of encoded data slices; and

one or more properties of the set of storage units.

10. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

determine that storage of a set of storage units has been expanded;

determine, for a plurality of sets of a desired number of encoded data slices, updated storage properties based on the storage expansion of the set of storage units, wherein a number of storage units in the set of storage units exceeds a number of encoded data slices of a set of the desired number of encoded data slices of the plurality of sets of the desired number of encoded data slices; and

for each set of the desired number of encoded data slices of at least some of the plurality of sets of the desired number of encoded data slices;

perform a scoring function using the updated storage properties to produce a per set storage resultant;

identify a sub set of storage units of the set of storage units for storing the set of the desired number of encoded data slices based on the per set storage resultant; and

store the set of the desired number of encoded data slices in the sub set of storage units.

11. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to perform the scoring function further by:

computing a storage value for each storage unit of the set of storage units based on the updated storage properties to produce a set of storage values; and

ordering the set of storage values in accordance with the scoring function to produce an ordered set of storage values, wherein the per set storage resultant includes the ordered set of storage values.

12. The non-transitory computer readable storage medium of claim 11, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to identify the sub set of storage units by:

selecting a grouping of the ordered set of storage values in accordance with the scoring function; and

corresponding the sub set of storage units to the storage units having a storage value in the grouping of the ordered set of storage values.

13. The non-transitory computer readable storage medium of claim 10, wherein the set of the desired number of encoded data slices further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

for an encoded data slice of the set of the desired number of encoded data slices;

determine whether storage location within the set of storage units has changed;

when the storage location within the set of storage units has changed;

store the encoded data slice at a new storage location within the sub set of storage units; and

delete the encoded data slices from an old storage location with the set of storage units.

14. The non-transitory computer readable storage medium of claim 13 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

when the storage location within the set of storage units has not changed, maintain storage of the encoded data slices at the storage location.

15. The non-transitory computer readable storage medium of claim 10, wherein the set of the desired number of encoded data slices comprises one of:

a set of a write threshold number of encoded data slices; and

a set of an information dispersal algorithm (IDA) width number of encoded data slices, wherein a decode threshold number of encoded data slices of the desired number of encoded data slices is required to rebuild a corresponding data segment of data.

16. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to determine that storage of a set of storage units has been expanded by:

determining that the number of storage units in the set of storage units has increased in value from a previous number of storage units in the set of storage units.

17. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to determine that storage of a set of storage units has been expanded by:

determining that one or more memory devices have been added to at least one of the storage units of the set of storage units.

18. The non-transitory computer readable storage medium of claim 10, wherein updated storage properties comprises:

information regarding the set of the desired number of encoded data slices; and

one or more properties of the set of storage units.