Deploying and growing a set of dispersed storage units at and by non-information dispersal algorithm (IDA) width multiples

• US 9,971,649 B2
• Filed: 09/30/2016
• Issued: 05/15/2018
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg R., Motwani, Manish, Resch, Jason K., Volvovski, Ilya
• Primary Examiner(s): Chery, Mardochee
• Application Number:
  US15/283,196
• Publication Number:
  US 20170123698A1
• Time in Patent Office: 592 Days
• Field of Search: 711171
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 43/0864 Round trip delays
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods and apparatus for use in a dispersed storage network (DSN) to deploy and grow a set of dispersed storage (DS) units for use in the DSN memory. In an example of operation, a DS client module assigns one or more additional DS units to a storage set to form a new storage set, where data is encoded in the DSN utilizing a dispersed storage error encoding function in accordance with an information dispersal algorithm (IDA) width. For each encoded data slice stored in the existing storage set, the DS client module utilizes a distributed agreement protocol function to select a storage unit of the new storage set for storage of the encoded data slice.

Claims

1. A method of growing a dispersed storage network, the dispersed storage network including a first set of dispersed storage units, wherein a first dispersed storage unit of the first set of dispersed storage units stores a first encoded data slice and a second encoded data slice and wherein the first encoded data slice and the second encoded data slice originate from a first data source, the method comprising:

assigning one or more additional dispersed storage units to the dispersed storage network including the first set of dispersed storage units to form a second set of dispersed storage units the second set of dispersed storage units including the first set of dispersed storage units and the one or more additional dispersed storage units;

reallocating the first encoded data slice from the first dispersed storage unit to at least one of the one or more additional dispersed storage units of the second set of dispersed storage units that does not presently store the first encoded data slice; and

facilitating migration of the first encoded data slice from the first dispersed storage unit to the at least one of the one or more additional dispersed storage units of the second set of dispersed storage units that does not presently store the first encoded data slice.

2. The method of claim 1, wherein the dispersed storage units in the first set of dispersed storage units are fewer than an information dispersal algorithm width number.

3. The method of claim 1, wherein assigning one or more additional dispersed storage units to the dispersed storage network comprises determining a number of additional dispersed storage units.

4. The method of claim 3, wherein assigning one or more additional dispersed storage units to the dispersed storage network comprises identifying the one or more additional dispersed storage units.

5. The method of claim 4, further comprising selecting the one or more additional dispersed storage units identified for assignment.

6. The method of claim 3, wherein determining the number of additional dispersed storage units is based on one or more of a predetermination, an estimated future storage requirements and existing storage utilization levels.

7. The method of claim 1, wherein assigning one or more additional dispersed storage units to the dispersed storage network uses a distributed agreement protocol.

8. The method of claim 7, wherein the distributed agreement protocol updates first weights for dispersed storage units of the first set of dispersed storage units and establishes second weights for the one or more additional dispersed storage units.

9. The method of claim 1, wherein facilitating migration comprises sending the first encoded data slice to a dispersed storage computing device.

10. A first dispersed storage unit of a first set of dispersed storage units for use in a dispersed storage network, the first dispersed storage unit comprising:

a communications interface;

a memory; and

a processor;

wherein the memory includes a first encoded data slice and a second encoded data wherein the first encoded data slice and the second encoded data slice originate from a first data source and wherein the memory further includes instructions for causing the processor to;

assign one or more additional dispersed storage units to the dispersed storage network including the first set of dispersed storage units to form a second set of dispersed storage units the second set of dispersed storage units including the first set of dispersed storage units and the one or more additional dispersed storage units;

reallocate the first encoded data slice from the first dispersed storage unit to at least one of the one or more additional dispersed storage units of the second set of dispersed storage units that does not presently store the first encoded data slice; and

facilitate migration of the first encoded data slice from the first dispersed storage unit to the at least one of the one or more additional dispersed storage units of the second set of dispersed storage units that does not presently store the first encoded data slice.

11. The first dispersed storage unit of claim 10, wherein the dispersed storage units in the first set of dispersed storage units are fewer than an information dispersal algorithm width number.

12. The first dispersed storage unit of claim 10, wherein the memory further comprises instructions for causing the processor to determine a number of additional dispersed storage units.

13. The first dispersed storage unit of claim 12, wherein the memory further comprises instructions for causing the processor identify the one or more additional dispersed storage units.

14. The first dispersed storage unit of claim 13, wherein the memory further comprises instructions for causing the processor to select the one or more additional dispersed storage units for assignment.

15. The first dispersed storage unit of claim 12, wherein the instructions for causing the processor to determine a number of additional dispersed storage units uses one or more of a predetermination, estimated future storage requirements and existing storage utilization levels.

16. The first dispersed storage unit of claim 10, wherein the instructions for causing the processor to assign one or more additional dispersed storage units to the dispersed storage network uses a distributed agreement protocol.

17. The first dispersed storage unit of claim 16, wherein the distributed agreement protocol is operable to update first weights for dispersed storage units of the first set of dispersed storage units and operable to establish second weights for the one or more additional dispersed storage units.

18. The first dispersed storage unit of claim 10, wherein the memory further comprises instructions for causing the processor to send the first encoded data slice to a dispersed storage computing device.

19. A dispersed storage network comprising:

a first set of dispersed storage units including a first dispersed storage unit;

the first dispersed storage unit including;

a communications interface;

a memory; and

a processor;

wherein the memory includes a first encoded data slice and a second encoded data wherein the first encoded data slice and the second encoded data slice originate from a first data source and wherein the memory further includes instructions for causing the processor to;

assign one or more additional dispersed storage units to the dispersed storage network including the first set of dispersed storage units to form a second set of dispersed storage units the second set of dispersed storage units including the first set of dispersed storage units and the one or more additional dispersed storage units;

reallocate the first encoded data slice from the first dispersed storage unit to at least one of the one or more additional dispersed storage units of the second set of dispersed storage units that does not presently store the first encoded data slice; and

facilitate migration of the first encoded data slice from the first dispersed storage unit to the at least one of the one or more additional dispersed storage units of the second set of dispersed storage units that does not presently store the first encoded data slice.

20. The dispersed storage network of claim 19, wherein the dispersed storage units in the first set of dispersed storage units are fewer than an information dispersal algorithm width number.