Detecting a utilization imbalance between dispersed storage network storage units

• US 9,836,352 B2
• Filed: 07/10/2014
• Issued: 12/05/2017
• Est. Priority Date: 11/25/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Cilfone, Bart, Leggette, Wesley, Gladwin, S. Christopher
• Primary Examiner(s): Mackall, Larry T
• Application Number:
  US14/327,585
• Publication Number:
  US 20140325307A1
• Time in Patent Office: 1,244 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G11B 5/09 Digital recording
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module of a dispersed storage network (DSN) obtaining utilization information regarding a plurality of storage units of the DSN, where first and second sets of storage units support a first logical storage vault. The method continues with the processing module detecting a utilization imbalance between a first storage unit of the first set of storage units and a second storage unit of the second set of storage units based on the utilization information, where the first and second storage units are not a common storage unit. The method continues with the processing module executing a data storage function regarding the first logical storage vault based on the utilization imbalance.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

obtaining utilization information regarding a plurality of storage units of the DSN, wherein first and second sets of storage units of the plurality of storage units support a first logical storage vault, wherein the first and second sets of storage units includes a common storage unit, wherein the utilization information of a storage unit of the plurality of storage units includes a listing of logical storage vaults supported by the storage unit and storage capacity consumed for each logical storage vault listed in the listing of logical storage vaults, wherein the first logical storage vault stores a plurality of encoded data objects, and wherein an encoded data object of the plurality of encoded data objects consists of a data object being dispersed storage error encoded into a plurality of sets of encoded data slices;

detecting a utilization imbalance between a first storage unit of the first set of storage units and a second storage unit of the second set of storage units based on the utilization information, wherein the first and second storage units are not the common storage unit; and

executing a data storage function regarding the first logical storage vault based on the utilization imbalance.

2. The method of claim 1, wherein the utilization information further comprises:

available storage capacity of the storage unit; and

storage capacity trend data.

3. The method of claim 1, wherein the obtaining the utilization information comprises one of:

querying the plurality of storage units for the utilization information;

receiving the utilization information from the plurality of storage units;

or

retrieving the utilization information.

4. The method of claim 1, wherein the executing the data storage function comprises:

receiving a write request as the data storage function;

selecting the first set of storage units or the second set of storage units based on the utilization imbalance for storing a first data object associated with the write request to produce a selected set of storage units; and

facilitating storage of the first data object in the selected set of storage units.

5. The method of claim 4, wherein the facilitating storage of the first data object comprises:

dispersed storage error encoding the first data object to produce a first plurality of sets of encoded data slices; and

sending write commands to the storage units of the selected set of storage units regarding storage of the first plurality of sets of encoded data slices.

6. The method of claim 1, wherein the executing the data storage function comprises:

determining that utilization of the first storage unit has reached a utilization threshold of the first storage unit;

determining that utilization of the second storage unit is below a utilization threshold of the second storage unit;

identifying encoded data slices of selected sets of encoded data slices of the plurality of sets of encoded data slices to migrate stored by the first storage unit; and

migrating the encoded data slices of selected sets of encoded data slices from the first storage unit to the second storage unit.

7. The method of claim 6 further comprises:

updating storage location information of the encoded data slices of selected sets of encoded data slices as currently being stored by the second storage unit.

8. A nontransitory computer readable memory device comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

obtain utilization information regarding a plurality of storage units of the DSN, wherein first and second sets of storage units of the plurality of storage units support a first logical storage vault, wherein the first and second sets of storage units includes a common storage unit, wherein the utilization information of a storage unit of the plurality of storage units includes a listing of logical storage vaults supported by the storage unit and storage capacity consumed for each logical storage vault listed in the listing of logical storage vaults, wherein the first logical storage vault stores a plurality of encoded data objects, and wherein an encoded data object of the plurality of encoded data objects consists of a data object being dispersed storage error encoded into a plurality of sets of encoded data slices;

detect a utilization imbalance between a first storage unit of the first set of storage units and a second storage unit of the second set of storage units based on the utilization information, wherein the first and second storage units are not the common storage unit; and

execute a data storage function regarding the first logical storage vault based on the utilization imbalance.

9. The nontransitory computer readable memory device of claim 8, wherein the utilization information further comprises:

available storage capacity of the storage unit; and

storage capacity trend data.

10. The nontransitory computer readable memory device of claim 8, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to obtain the utilization information by one of:

querying the plurality of storage units for the utilization information;

receiving the utilization information from the plurality of storage units;

or

retrieving the utilization information.

11. The nontransitory computer readable memory device of claim 8, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to execute the data storage function by:

receiving a write request as the data storage function;

selecting the first set of storage units or the second set of storage units based on the utilization imbalance for storing a first data object associated with the write request to produce a selected set of storage units; and

facilitating storage of the first data object in the selected set of storage units.

12. The nontransitory computer readable memory device of claim 11, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to facilitate storage of the first data object by:

dispersed storage error encoding the first data object to produce a first plurality of sets of encoded data slices; and

sending write commands to the storage units of the selected set of storage units regarding storage of the first plurality of sets of encoded data slices.

13. The nontransitory computer readable memory device of claim 8, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to execute the data storage function by:

determining that utilization of the first storage unit has reached a utilization threshold of the first storage unit;

determining that utilization of the second storage unit is below a utilization threshold of the second storage unit;

identifying encoded data slices of selected sets of encoded data slices of the plurality of sets of encoded data slices to migrate stored by the first storage unit; and

migrating the encoded data slices of selected sets of encoded data slices from the first storage unit to the second storage unit.

14. The nontransitory computer readable memory device of claim 13 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

update storage location information of the encoded data slices of selected sets of encoded data slices as currently being stored by the second storage unit.

15. A computing device of a dispersed storage network (DSN), the computing device comprises:

an interface; and

a processing module operably coupled to the interface, wherein the processing module is operable to;

obtain utilization information regarding a plurality of storage units of the DSN, wherein first and second sets of storage units of the plurality of storage units support a first logical storage vault, wherein the first and second sets of storage units includes a common storage unit, wherein the utilization information of a storage unit of the plurality of storage units includes a listing of logical storage vaults supported by the storage unit and storage capacity consumed for each logical storage vault listed in the listing of logical storage vaults, wherein the first logical storage vault stores a plurality of encoded data objects, and wherein an encoded data object of the plurality of encoded data objects consists of a data object being dispersed storage error encoded into a plurality of sets of encoded data slices;

detect a utilization imbalance between a first storage unit of the first set of storage units and a second storage unit of the second set of storage units based on the utilization information, wherein the first and second storage units are not the common storage unit; and

execute a data storage function regarding the first logical storage vault based on the utilization imbalance.

16. The computing device of claim 15, wherein the utilization information further comprises:

available storage capacity of the storage unit; and

storage capacity trend data.

17. The computing device of claim 15, wherein the processing module obtains the utilization information by one of:

querying the plurality of storage units for the utilization information;

receiving, via the interface, the utilization information from the plurality of storage units;

or

retrieving the utilization information.

18. The computing device of claim 15, wherein the processing module executes the data storage function by:

receiving, via the interface, a write request as the data storage function;

selecting the first set of storage units or the second set of storage units based on the utilization imbalance for storing a first data object associated with the write request to produce a selected set of storage units; and

facilitating storage of the first data object in the selected set of storage units.

19. The computing device of claim 18, wherein the processing module facilitates storage of the first data object by:

dispersed storage error encoding the first data object to produce a first plurality of sets of encoded data slices; and

sending, via the interface, write commands to the storage units of the selected set of storage units regarding storage of the first plurality of sets of encoded data slices.

20. The computing device of claim 15, wherein the processing module executes the data storage function by:

determining that utilization of the first storage unit has reached a utilization threshold of the first storage unit;

determining that utilization of the second storage unit is below a utilization threshold of the second storage unit;

identifying encoded data slices of selected sets of encoded data slices of the plurality of sets of encoded data slices to migrate stored by the first storage unit; and

migrating the encoded data slices of selected sets of encoded data slices from the first storage unit to the second storage unit.

21. The storage unit of claim 20 further comprises:

the processing module is further operable to;

update storage location information of the encoded data slices of selected sets of encoded data slices as currently being stored by the second storage unit.