Optimizing secondary storage in a dispersed storage network

• US 10,042,706 B2
• Filed: 09/30/2016
• Issued: 08/07/2018
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K.
• Primary Examiner(s): ABRAHAM, ESAW T
• Application Number:
  US15/281,728
• Publication Number:
  US 20170123908A1
• Time in Patent Office: 676 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 43/0864 Round trip delays
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by one or more processing modules of a distributed storage network includes encoding a data segment of data using a dispersed storage error coding function to produce a write threshold number of encoded data slices and identifying primary storage units available for storing the write threshold number of encoded data slices. The method further includes issuing a write slice request that includes a slice name and an encoded data slice corresponding to each of the available primary storage units. For those primary storage units that are unavailable the method continues by selecting a storage approach for storing an encoded data slice and selecting a foster storage unit. The method continues by issuing a write imposter slice request to each selected foster storage unit, where the imposter encoded data slice includes an information dispersal algorithm (IDA) index corresponding to the associated unavailable storage unit.

Claims

1. A method that is executed by one or more processing modules of one or more computing devices, the method comprises:

encoding a data segment of data using a dispersed storage error coding function to produce a write threshold number of encoded data slices;

identifying, by a distributed storage and task (DST) processing unit which of a plurality of primary storage units is available for storing the write threshold number of encoded data slices;

for those primary storage units of the plurality of primary storage units that are available, issuing a write slice request that includes a slice name corresponding to each of the available primary storage units and an encoded data slice corresponding to each of the available primary storage units;

for those primary storage units of the plurality of primary storage units that are unavailable, selecting a storage approach for storing an encoded data slice associated with the unavailable primary storage units;

for each of the primary storage units of the primary storage units that are unavailable, further selecting a foster storage unit in accordance with the selected storage approach; and

issuing a write imposter slice request to each selected foster storage unit, wherein the request includes a slice name corresponding to the selected foster storage unit and an imposter encoded data slice corresponding to an unavailable primary storage unit.

2. The method of claim 1, wherein the selecting a storage approach is based on at least one of a storage reliability requirement, a storage capacity utilization level of the set of storage units, a dispersed storage network system activity level, a storage unit write performance level, a predetermination, system registry information, or a request.

3. The method of claim 1, wherein one or more of the selected foster storage units is located at the same site as an unavailable primary storage unit.

4. The method of claim 1 further comprises:

for those primary storage units of the plurality of primary storage units that are available, including an information dispersal algorithm (IDA) index of the primary storage unit with the encoded data slice.

5. The method of claim 1, wherein the selected foster storage units are selected randomly from available storage units.

6. The method of claim 1, wherein the write imposter slice request is generated by the DST processing unit.

7. The method of claim 1, wherein the identifying, by the DST processing module selects the which of a plurality of primary storage units is available for storing the write threshold number of encoded data slices is based on at least one of a random selection approach, a predetermination, a storage unit performance level, a storage unit available capacity level, or a system registry information where the IDA width of the set of storage units is greater than the write threshold number.

8. The method of claim 1, wherein the selecting a storage approach is based on higher storage capacity being desired and a higher storage capacity storage unit being available.

9. The method of claim 1, wherein the selecting a storage approach is based on maintaining even distribution of encoded data slices across the primary storage units.

10. The method of claim 1, wherein the imposter encoded data slice includes an information dispersal algorithm (IDA) index corresponding to the associated unavailable primary storage unit.

11. A computing device comprises:

an interface; and

a processing module, when operable within the computing device, causes the computing device to;

encode a data segment of data using a dispersed storage error coding function to produce a write threshold number of encoded data slices;

identify, which of a plurality of primary storage units is available for storing the write threshold number of encoded data slices;

for those primary storage units of the plurality of primary storage units that are available, issue a write slice request that includes a slice name corresponding to each of the primary storage units and an encoded data slice corresponding to each of the primary storage units;

for those primary storage units of the plurality of primary storage units that are unavailable, select a storage approach for storing an encoded data slice associated with the unavailable primary storage units;

for each of the primary storage units of the primary storage units that are unavailable, further select a foster storage unit in accordance with the selected storage approach; and

issue a write imposter slice request to each selected foster storage unit, wherein the request includes a slice name corresponding to the selected foster storage unit and an imposter encoded data slice corresponding to an unavailable storage unit, where the imposter encoded data slice includes an information dispersal algorithm (IDA) index corresponding to the associated unavailable storage unit.

12. The computing device of claim 11, wherein the processing module functions to select a storage approach based on at least one of a storage reliability requirement, a storage capacity utilization level of the set of storage units, a dispersed storage network system activity level, a storage unit write performance level, a predetermination, system registry information, or a request.

13. The computing device of claim 11, wherein the processing module functions to select foster storage units located at the same site as an unavailable primary storage unit.

14. The computing device of claim 11, wherein the processing module functions to include an information dispersal algorithm (IDA) index of the primary storage unit with the encoded data slice for those primary storage units of the plurality of primary storage units that are available.

15. The computing device of claim 11, wherein the processing module functions to select foster storage units randomly from available storage units.

16. The computing device of claim 11, wherein the processing module functions to generates the write imposter slice request.

17. The computing device of claim 11, wherein the processing module functions to select which of a plurality of primary storage units is available for storing the write threshold number of encoded data slices based on at least one of a random selection approach, a predetermination, a storage unit performance level, a storage unit available capacity level, or a system registry information where the IDA width of the set of storage units is greater than the write threshold number.

18. The computing device of claim 11, wherein the processing module functions to select a storage approach based on higher storage capacity being desired and a higher storage capacity storage unit being available.

19. The computing device of claim 11, wherein the processing module functions to select a storage approach based on maintaining even distribution of encoded data slices across the primary storage units.