Expanding information dispersal algorithm width without rebuilding through imposter slices

• US 10,241,864 B2
• Filed: 09/30/2016
• Issued: 03/26/2019
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Baptist, Andrew D., Dhuse, Greg R., Khadiwala, Ravi V., Resch, Jason K., Volvovski, Ilya, Wozniak, Ethan S.
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): Li, Hewy H
• Application Number:
  US15/282,888
• Publication Number:
  US 20170123697A1
• Time in Patent Office: 907 Days
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 43/0864 Round trip delays
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method and system for expanding a set of storage units. Expanding a set of storage units includes facilitating storage of a set of encoded data slices in a set of storage units within a first time frame. When detecting expansion of the set of storage units with new storage units within a second timeframe, primary storage units of the expanded set of storage units based are identified based on a write threshold value associated with the expanded set of storage units. For each new primary storage unit, storage of an imposter encoded data slice sourced from another storage unit that is not a primary storage unit but holds an encoded data slice of the set of encoded data slices is facilitated. The other storage unit then deletes the imposter encoded data slice.

Claims

1. A method of accessing data in a dispersed storage network, the dispersed storage network including an initial set of dispersed storage units comprising:

at a first time, storing in each respective dispersed storage unit of the initial set of dispersed storage units a respective encoded data slice of a set of encoded data slices;

at a second time detecting an expansion of the initial set of dispersed storage units to include new dispersed storage units, the expansion resulting in an expanded set of dispersed storage units, wherein the expansion of the initial set of dispersed storage units is based on increasing an information dispersal algorithm (IDA) width;

identifying primary storage units of the expanded set of dispersed storage units; and

for each respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units, storing the respective encoded data slice of one of the respective dispersed storage units of the initial set of dispersed storage units that is not one of the primary storage units.

2. The method of claim 1, wherein the expansion of the initial set of dispersed storage units is based on increasing a write threshold.

3. The method of claim 1, wherein the step of identifying primary storage units of the expanded set of dispersed storage units is based on a write threshold associated with the expanded set of dispersed storage units.

4. The method of claim 1, wherein step of identifying primary storage units of the expanded set of dispersed storage units is based on one or more of interpreting system registration information, a storage unit performance level, an available storage capacity level, a predetermination and a random selection.

5. The method of claim 1, wherein the step of identifying primary storage units of the expanded set of dispersed storage units is performed by one or more of the respective dispersed storage units of the initial set of dispersed storage units that are not primary storage units.

6. The method of claim 1, wherein the respective encoded data slice of the respective dispersed storage unit of the initial set of dispersed storage units that is not one of the primary storage units that is stored in the respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units is embedded with an IDA index of the respective dispersed storage unit of the initial set of the dispersed storage units that is not one of the primary storage units.

7. The method of claim 6, wherein the respective encoded data slice stored in the respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units includes a slice name associated with the respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units.

8. The method of claim 1, wherein the dispersed storage network is transitioned to a trimmed writes configuration.

9. A dispersed storage processing unit for use in a dispersed storage network, the dispersed storage network including an initial set of dispersed storage units, the dispersed storage processing unit comprising:

a communications interface;

a memory; and

a processor;

wherein the memory includes a set of encoded data slices and wherein the memory further includes instructions for causing the processor to;

at a first time, store in each respective dispersed storage unit of the initial set of dispersed storage units a respective encoded data slice of the set of encoded data slices;

at a second time detect an expansion of the initial set of dispersed storage units to include new dispersed storage units, the expansion resulting in an expanded set of dispersed storage units, wherein the expansion of the initial set of dispersed storage units is based on increasing a write threshold;

identify primary storage units of the expanded set of dispersed storage units; and

for each respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units, store the respective encoded data slice of one of the respective dispersed storage units of the initial set of dispersed storage units that is not one of the primary storage units.

10. The dispersed storage processing unit of claim 9, wherein the expansion of the initial set of dispersed storage units is based on increasing an information dispersal algorithm (IDA) Width.

11. The dispersed storage processing unit of claim 9, wherein the instructions for causing the processor to identify primary storage units of the expanded set of dispersed storage units are based on a write threshold associated with the expanded set of dispersed storage units.

12. The dispersed storage processing unit of claim 9, wherein the instructions for causing the processor to identify primary storage units of the expanded set of dispersed storage units are based on one or more of interpreting system registration information, a storage unit performance level, an available storage capacity level, a predetermination and a random selection.

13. The dispersed storage processing unit of claim 9, wherein the respective encoded data slice of one of the respective dispersed storage units of the initial set of dispersed storage units that is not one of the primary storage units is embedded with an IDA index of the respective dispersed storage unit of the initial set of the dispersed storage units that is not one of the primary storage units.

14. The dispersed storage processing unit of claim 13, wherein the respective encoded data slice of one of the respective dispersed storage units of the initial set of dispersed storage units that is not one of the primary storage units includes a slice name associated with the respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units.

15. The dispersed storage processing unit of claim 9, wherein the dispersed storage network utilizes a trimmed writes configuration.

16. A dispersed storage network comprising:

an initial set of dispersed storage units; and

a dispersed storage processing unit, the dispersed storage processing unit including;

a first communications interface;

a first memory; and

a first processor;

wherein the first memory includes a set of encoded data slices and wherein the first memory further includes instructions for causing the first processor to;

at a first time, store in each respective dispersed storage unit of the initial set of dispersed storage units a respective encoded data slice of the set of encoded data slices;

at a second time detect an expansion of the initial set of dispersed storage units to include new dispersed storage units, the expansion resulting in an expanded set of dispersed storage units; and

for each respective primary storage unit of the expanded set of dispersed storage units that is one of the new dispersed storage units, store the respective encoded data slice of one of the respective dispersed storage units of the initial set of dispersed storage units that is not one of the primary storage units;

wherein one or more of the dispersed storage units of the initial set of dispersed storage units include;

a second communications interface;

a second memory; and

a second processor; and

wherein the second memory includes instructions for causing the second processor to identify primary storage units of the expanded set of dispersed storage units based on a write threshold associated with the expanded set of dispersed storage units.

17. The dispersed storage network of claim 16, wherein the dispersed storage network utilizes a trimmed writes configuration.