Recovering an encoded data slice in a dispersed storage network

• US 9,690,520 B2
• Filed: 05/26/2015
• Issued: 06/27/2017
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Grube, Gary W., Resch, Jason K.
• Primary Examiner(s): CHAUDRY, MUJTABA M
• Application Number:
  US14/721,615
• Publication Number:
  US 20150378822A1
• Time in Patent Office: 763 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module of a dispersed storage network (DSN) receiving a DSN retrieval request regarding a data object and performing a scoring function using properties of the DSN retrieval request and properties of DSN memory of the DSN to produce a storage scoring resultant. The method continues with the processing module identifying a set of primary storage units based on the storage scoring resultant and sending a set of retrieval requests to the set of primary storage units. When a primary storage unit does not provide a favorable response, using the storage scoring resultant to identify an alternative storage unit. When the alternative storage unit is identified, sending a corresponding retrieval request to the alternative storage unit.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a DSN retrieval request regarding a data object;

performing a scoring function using one or more properties of the DSN retrieval request and one or more properties of DSN memory of the DSN to produce a storage scoring resultant, wherein the DSN memory includes a plurality of storage units that are logically arranged into a plurality of storage pools;

identifying a set of primary storage units of the plurality of storage units based on the storage scoring resultant;

sending a set of retrieval requests to the set of primary storage units regarding the DSN retrieval request;

when a primary storage unit of the set of primary storage units does not provide a favorable response to a corresponding one of the set of retrieval requests regarding a corresponding portion of the data object, using the storage scoring resultant to identify an alternative storage unit of the plurality of storage units regarding the corresponding portion of the data object; and

when the alternative storage unit is identified, sending the corresponding one of the set of retrieval requests to the alternative storage unit.

2. The method of claim 1, wherein the performing the scoring function further comprises:

selecting a resource level; and

selecting the one or more properties of the DSN memory from a plurality of properties of the DSN memory based on the selected resource level.

3. The method of claim 2 further comprises:

calculating, based on the selected resource level, a plurality of storage values based on the one or more properties of the DSN retrieval request and the one or more properties of DSN memory; and

performing a ranking function on the plurality of storage values to produce the storage scoring resultant.

4. The method of claim 1, wherein the performing the scoring function further comprises:

selecting a storage pool level indication as a resource level;

selecting a storage pool identifier and a storage pool weighting factor for each of the plurality of storage pools to produce a plurality of storage pool identifiers and a plurality of storage pool weighting factors, wherein the one or more properties of DSN memory includes the plurality of storage pool identifiers and the plurality of storage pool weighting factors;

selecting a source name of the DSN retrieval request as the one or more properties of the DSN retrieval request;

performing a series of functions on the source name based on the plurality of storage pool identifiers and the plurality of storage pool weighting factors to produce a plurality of storage values; and

performing a ranking function on the plurality of storage values to produce the storage scoring resultant.

5. The method of claim 4, wherein a series of the series of functions comprises:

a deterministic function of the source name and one of the storage pool identifiers to produce an interim result;

a normalizing function of the interim result to produce a normalized interim result; and

a scoring function of the normalized interim result and a corresponding one of the storage pool weighting factors to produce a storage value of the plurality of storage values.

6. The method of claim 1, wherein the performing the scoring function further comprises:

selecting a storage unit level indication as a resource level;

selecting a storage site-storage unit identifier and a storage site-storage weighting factor for each of the plurality of storage units to produce a plurality of storage site-storage unit identifiers and a plurality of storage site-storage unit weighting factors, wherein the one or more properties of DSN memory includes the plurality of storage site-storage unit identifiers and the plurality of storage site-storage unit weighting factors;

selecting a source name of the DSN retrieval request as the one or more properties of the DSN retrieval request;

performing a series of functions on the source name based on the plurality of storage site-storage unit identifiers and the plurality of storage site-storage unit weighting factors to produce a plurality of storage values; and

performing a ranking function on the plurality of storage value to produce the storage scoring resultant.

7. The method of claim 1, wherein the DSN retrieval request comprises one of:

a read request for reading the data object from the DSN memory, wherein the favorable response to the corresponding one of the set of retrieval requests includes sending the corresponding portion of the data object;

a list request for storage information regarding the data object, wherein the favorable response to a corresponding one of the set of retrieval requests includes sending storage information regarding the corresponding portion of the data object;

an individual read request regarding the corresponding portion of the data object, wherein the favorable response includes sending the corresponding portion of the data object and wherein the set of primary storage units only includes the primary storage unit; and

an individual list request regarding the corresponding portion of the data object, wherein the favorable response includes sending the storage information of the corresponding portion of the data object and wherein the set of primary storage units only includes the primary storage unit.

8. The method of claim 1 further comprises:

when the alternative storage unit is not identified, issuing a rebuilding function for the corresponding portion of the data object, wherein a rebuilt corresponding portion of the data object is to be stored in the alternative storage unit.

9. The method of claim 8 further comprises:

determining whether the primary storage unit is available for storing the rebuilt corresponding portion of the data object; and

when the primary storage unit is available for storing the rebuilt corresponding portion of the data object, transferring the rebuilt corresponding portion of the data object from the alternative storage unit to the primary storage unit.

10. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

receive a DSN retrieval request regarding a data object;

perform a scoring function using one or more properties of the DSN retrieval request and one or more properties of DSN memory of the DSN to produce a storage scoring resultant, wherein the DSN memory includes a plurality of storage units that are logically arranged into a plurality of storage pools;

identify a set of primary storage units of the plurality of storage units based on the storage scoring resultant;

send a set of retrieval requests to the set of primary storage units regarding the DSN retrieval request;

when a primary storage unit of the set of primary storage units does not provide a favorable response to a corresponding one of the set of retrieval requests regarding a corresponding portion of the data object, use the storage scoring resultant to identify an alternative storage unit of the plurality of storage units regarding the corresponding portion of the data object; and

when the alternative storage unit is identified, send the corresponding one of the set of retrieval requests to the alternative storage unit.

11. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to further perform the scoring function by:

selecting a resource level; and

selecting the one or more properties of the DSN memory from a plurality of properties of the DSN memory based on the selected resource level.

12. The non-transitory computer readable storage medium of claim 11 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

calculate, based on the selected resource level, a plurality of storage values based on the one or more properties of the DSN retrieval request and the one or more properties of DSN memory; and

perform a ranking function on the plurality of storage values to produce the storage scoring resultant.

13. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to further perform the scoring function by:

selecting a storage pool level indication as a resource level;

selecting a storage pool identifier and a storage pool weighting factor for each of the plurality of storage pools to produce a plurality of storage pool identifiers and a plurality of storage pool weighting factors, wherein the one or more properties of DSN memory includes the plurality of storage pool identifiers and the plurality of storage pool weighting factors;

selecting a source name of the DSN retrieval request as the one or more properties of the DSN retrieval request;

performing a series of functions on the source name based on the plurality of storage pool identifiers and the plurality of storage pool weighting factors to produce a plurality of storage values; and

performing a ranking function on the plurality of storage values to produce the storage scoring resultant.

14. The non-transitory computer readable storage medium of claim 13, wherein a series of the series of functions comprises:

a deterministic function of the source name and one of the storage pool identifiers to produce an interim result;

a normalizing function of the interim result to produce a normalized interim result; and

a scoring function of the normalized interim result and a corresponding one of the storage pool weighting factors to produce a storage value of the plurality of storage values.

15. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to further perform the scoring function by:

selecting a storage unit level indication as a resource level;

selecting a storage site-storage unit identifier and a storage site-storage weighting factor for each of the plurality of storage units to produce a plurality of storage site-storage unit identifiers and a plurality of storage site-storage unit weighting factors, wherein the one or more properties of DSN memory includes the plurality of storage site-storage unit identifiers and the plurality of storage site-storage unit weighting factors;

selecting a source name of the DSN retrieval request as the one or more properties of the DSN retrieval request;

performing a series of functions on the source name based on the plurality of storage site-storage unit identifiers and the plurality of storage site-storage unit weighting factors to produce a plurality of storage values; and

performing a ranking function on the plurality of storage value to produce the storage scoring resultant.

16. The non-transitory computer readable storage medium of claim 10, wherein the DSN retrieval request comprises one of:

a read request for reading the data object from the DSN memory, wherein the favorable response to the corresponding one of the set of retrieval requests includes sending the corresponding portion of the data object;

a list request for storage information regarding the data object, wherein the favorable response to a corresponding one of the set of retrieval requests includes sending storage information regarding the corresponding portion of the data object;

an individual read request regarding the corresponding portion of the data object, wherein the favorable response includes sending the corresponding portion of the data object and wherein the set of primary storage units only includes the primary storage unit; and

an individual list request regarding the corresponding portion of the data object, wherein the favorable response includes sending the storage information of the corresponding portion of the data object and wherein the set of primary storage units only includes the primary storage unit.

17. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

when the alternative storage unit is not identified, issue a rebuilding function for the corresponding portion of the data object, wherein a rebuilt corresponding portion of the data object is to be stored in the alternative storage unit.

18. The non-transitory computer readable storage medium of claim 17 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine whether the primary storage unit is available for storing the rebuilt corresponding portion of the data object; and

when the primary storage unit is available for storing the rebuilt corresponding portion of the data object, transfer the rebuilt corresponding portion of the data object from the alternative storage unit to the primary storage unit.

19. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

receive a DSN retrieval request regarding a data object;

perform a scoring function using one or more properties of the DSN retrieval request and one or more properties of DSN memory of the DSN to produce a storage scoring resultant, wherein the DSN memory includes a plurality of storage units that are logically arranged into a plurality of storage pools;

identify a set of primary storage units of the plurality of storage units based on the storage scoring resultant;

send, via the interface, a set of retrieval requests to the set of primary storage units regarding the DSN retrieval request;

when a primary storage unit of the set of primary storage units does not provide a favorable response to a corresponding one of the set of retrieval requests regarding a corresponding portion of the data object, use the storage scoring resultant to identify an alternative storage unit of the plurality of storage units regarding the corresponding portion of the data object; and

when the alternative storage unit is identified, send, via the interface, the corresponding one of the set of retrieval requests to the alternative storage unit.

20. The computing device of claim 19, wherein the processing module functions to further perform the scoring function by:

selecting a resource level; and

selecting the one or more properties of the DSN memory from a plurality of properties of the DSN memory based on the selected resource level.

21. The computing device of claim 20, wherein the processing module further functions to:

calculate, based on the selected resource level, a plurality of storage values based on the one or more properties of the DSN retrieval request and the one or more properties of DSN memory; and

perform a ranking function on the plurality of storage values to produce the storage scoring resultant.

22. The computing device of claim 19, wherein the processing module functions to further perform the scoring function by:

selecting a storage pool level indication as a resource level;

selecting a storage pool identifier and a storage pool weighting factor for each of the plurality of storage pools to produce a plurality of storage pool identifiers and a plurality of storage pool weighting factors, wherein the one or more properties of DSN memory includes the plurality of storage pool identifiers and the plurality of storage pool weighting factors;

selecting a source name of the DSN retrieval request as the one or more properties of the DSN retrieval request;

performing a series of functions on the source name based on the plurality of storage pool identifiers and the plurality of storage pool weighting factors to produce a plurality of storage values; and

performing a ranking function on the plurality of storage values to produce the storage scoring resultant.

23. The computing device of claim 22, wherein a series of the series of functions comprises:

a deterministic function of the source name and one of the storage pool identifiers to produce an interim result;

a normalizing function of the interim result to produce a normalized interim result; and

a scoring function of the normalized interim result and a corresponding one of the storage pool weighting factors to produce a storage value of the plurality of storage values.

24. The computing device of claim 19, wherein the processing module functions to further perform the scoring function by:

selecting a storage unit level indication as a resource level;

selecting a storage site-storage unit identifier and a storage site-storage weighting factor for each of the plurality of storage units to produce a plurality of storage site-storage unit identifiers and a plurality of storage site-storage unit weighting factors, wherein the one or more properties of DSN memory includes the plurality of storage site-storage unit identifiers and the plurality of storage site-storage unit weighting factors;

selecting a source name of the DSN retrieval request as the one or more properties of the DSN retrieval request;

performing a series of functions on the source name based on the plurality of storage site-storage unit identifiers and the plurality of storage site-storage unit weighting factors to produce a plurality of storage values; and

performing a ranking function on the plurality of storage value to produce the storage scoring resultant.

25. The computing device of claim 19, wherein the DSN retrieval request comprises one of:

a read request for reading the data object from the DSN memory, wherein the favorable response to the corresponding one of the set of retrieval requests includes sending the corresponding portion of the data object;

a list request for storage information regarding the data object, wherein the favorable response to a corresponding one of the set of retrieval requests includes sending storage information regarding the corresponding portion of the data object;

an individual read request regarding the corresponding portion of the data object, wherein the favorable response includes sending the corresponding portion of the data object and wherein the set of primary storage units only includes the primary storage unit; and

an individual list request regarding the corresponding portion of the data object, wherein the favorable response includes sending the storage information of the corresponding portion of the data object and wherein the set of primary storage units only includes the primary storage unit.

26. The computing device of claim 19, wherein the processing module further functions to:

when the alternative storage unit is not identified, issue, via the interface, a rebuilding function for the corresponding portion of the data object, wherein a rebuilt corresponding portion of the data object is to be stored in the alternative storage unit.

27. The computing device of claim 26, wherein the processing module further functions to:

determine whether the primary storage unit is available for storing the rebuilt corresponding portion of the data object; and

when the primary storage unit is available for storing the rebuilt corresponding portion of the data object, transfer the rebuilt corresponding portion of the data object from the alternative storage unit to the primary storage unit.