Dispersed storage system with vault updating and methods for use therewith

• US 9,594,507 B2
• Filed: 08/05/2014
• Issued: 03/14/2017
• Est. Priority Date: 10/03/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Gladwin, S. Christopher
• Primary Examiner(s): MCMAHON, DANIEL F
• Application Number:
  US14/451,911
• Publication Number:
  US 20150100846A1
• Time in Patent Office: 952 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/1458 by checking the subject access rights
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2221/2149 Restricted operating environment
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0682 : Tape device
• CPC Class Codes: G06F 3/0689 : Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/4881 : Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: H04L 67/1001 : for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/145 : avoiding end of session, e.g. keep-alive, heartbeats, resumption message or wake-up for inactive or interrupted session

Abstract

A method for execution by one or more processing modules of a dispersed storage network (DSN) includes storing a set of encoded data slices in an original plurality of storage units of the DSN associated as a current generation of a storage vault. The method determines whether utilization of the original plurality of storage units is greater than a utilization threshold. When the utilization of the original plurality of storage units is greater than the utilization threshold, the original plurality of storage units are updated to include at least one additional storage unit and a proper subset of the original plurality of storage units associated as the current generation of a storage vault to generate an updated plurality of storage units associated as a next generation of the storage vault. The set of encoded data slices are stored in the updated plurality of storage units of the DSN associated as the next generation of the storage vault.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

distributedly storing a set of encoded data slices in an original plurality of storage units of the DSN associated as a current generation of a storage vault based on a current generation mapping of the set of encoded data slices that specifies the original plurality of storage units of the DSN, wherein the original plurality of storage units of the DSN includes a width dispersal parameter of a dispersed storage error encoding function by which the set of encoded data slices is generated;

determining whether utilization of the original plurality of storage units is greater than a utilization threshold;

when the utilization of the original plurality of storage units is greater than the utilization threshold, updating the original plurality of storage units to include at least one additional storage unit and a proper subset of the original plurality of storage units associated with the current generation of the storage vault to generate an updated plurality of storage units associated with a next generation of the storage vault, wherein the updated plurality of storage units includes more than a width dispersal parameter number of storage units; and

distributedly storing the set of encoded data slices in the updated plurality of storage units of the DSN associated as the next generation of the storage vault based on a next generation mapping of the set of encoded data slices that specifies the width dispersal parameter number of storage units of the updated plurality of storage units that includes the at least one additional storage unit and excludes at least one of the original plurality of storage units of the DSN.

2. The method of claim 1 wherein the original plurality of storage units include N storage units wherein N is an integer greater than 2 and wherein N−

1 of the original plurality of storage units are included in the updated plurality of storage units.

3. The method of claim 2 wherein N corresponds to a write threshold number of encoded data slices for a data segment stored in the DSN in at least one of the set of encoded data slices.

4. The method of claim 1 wherein the original plurality of storage units are associated with the current generation of the storage vault via a table.

5. The method of claim 1 wherein determining whether utilization of the original plurality of storage units is greater than the utilization threshold includes determining whether utilization of any one of the original plurality of storage units is greater than the utilization threshold.

6. The method of claim 1 wherein the proper subset of the original plurality of storage units is determined by including storage units of the original plurality of storage units having utilization that is less than the utilization threshold.

7. The method of claim 1 wherein the set of encoded data slices are stored in the updated plurality of storage units, such that none of the set of encoded data slices is stored in a same storage unit of the updated plurality of storage units used previously to store that same encoded data slice of the set of encoded data slices in the original plurality of storage units.

8. A dispersed storage and task (DST) processing unit comprises:

at least one processor, when operable within a computing device, that causes the computing device to;

distributedly store a set of encoded data slices in an original plurality of storage units of a dispersed storage network (DSN) associated as a current generation of a storage vault based on a current generation mapping of the set of encoded data slices that specifies the original plurality of storage units of the DSN, wherein the original plurality of storage units of the DSN includes a width dispersal parameter of a dispersed storage error encoding function by which the set of encoded data slices is generated;

determine whether utilization of the original plurality of storage units is greater than a utilization threshold;

when the utilization of the original plurality of storage units is greater than the utilization threshold, update the original plurality of storage units to include at least one additional storage unit and a proper subset of the original plurality of storage units associated with the current generation of the storage vault to generate an updated plurality of storage units associated with a next generation of the storage vault, wherein the updated plurality of storage units includes more than a width dispersal parameter number of storage units; and

distributedly store the set of encoded data slices in the updated plurality of storage units of the DSN associated as the next generation of the storage vault based on a next generation mapping of the set of encoded data slices that specifies the width dispersal parameter number of storage units of the updated plurality of storage units that includes the at least one additional storage unit and excludes at least one of the original plurality of storage units of the DSN.

9. The DST processing unit of claim 8 wherein the original plurality of storage units include N storage units wherein N is an integer greater than 2 and wherein N−

1 of the original plurality of storage units are included in the updated plurality of storage units.

10. The DST processing unit of claim 9 wherein N corresponds to a write threshold number of encoded data slices for a data segment stored in the DSN in at least one of the set of encoded data slices.

11. The DST processing unit of claim 8 wherein the original plurality of storage units are associated with the current generation of the storage vault via a table.

12. The DST processing unit of claim 8 wherein determining whether utilization of the original plurality of storage units is greater than the utilization threshold includes determining whether utilization of any one of the original plurality of storage units is greater than the utilization threshold.

13. The DST processing unit of claim 8 wherein the proper subset of the original plurality of storage units is determined by including storage units of the original plurality of storage units having utilization that is less than the utilization threshold.

14. The DST processing unit of claim 8 wherein the set of encoded data slices are stored in the updated plurality of storage units, such that none of the set of encoded data slices is stored in a same storage unit of the updated plurality of storage units used to previously store a same encoded data slice of the set of encoded data slices in the original plurality of storage units.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

distributedly store a set of encoded data slices in an original plurality of storage units of the DSN associated as a current generation of a storage vault based on a current generation mapping of the set of encoded data slices that specifies the original plurality of storage units of the DSN, wherein the original plurality of storage units of the DSN includes a width dispersal parameter of a dispersed storage error encoding function by which the set of encoded data slices is generated;

determine whether utilization of the original plurality of storage units is greater than a utilization threshold;

when the utilization of the original plurality of storage units is greater than the utilization threshold, update the original plurality of storage units to include at least one additional storage unit and a proper subset of the original plurality of storage units associated with the current generation of the storage vault to generate an updated plurality of storage units associated with a next generation of the storage vault, wherein updated plurality of storage units includes more than a width dispersal parameter number of storage units; and

distributedly store the set of encoded data slices in the updated plurality of storage units of the DSN associated as the next generation of the storage vault based on a next generation mapping of the set of encoded data slices that specifies the width dispersal parameter number of storage units of the updated plurality of storage units that includes the at least one additional storage unit and excludes at least one of the original plurality of storage units of the DSN.

16. The non-transitory computer readable storage medium of claim 15 wherein the original plurality of storage units include N storage units wherein N is an integer greater than 2 and wherein N−

1 of the original plurality of storage units are included in the updated plurality of storage units.

17. The non-transitory computer readable storage medium of claim 16 wherein N corresponds to a write threshold number of encoded data slices for a data segment stored in the DSN in at least one of the set of encoded data slices.

18. The non-transitory computer readable storage medium of claim 15 wherein determining whether utilization of the original plurality of storage units is greater than the utilization threshold includes determining whether utilization of any one of the original plurality of storage units is greater than the utilization threshold.

19. The non-transitory computer readable storage medium of claim 15 wherein the proper subset of the original plurality of storage units is determined by including storage units of the original plurality of storage units having utilization that is less than the utilization threshold.

20. The non-transitory computer readable storage medium of claim 15 wherein the set of encoded data slices are stored in the updated plurality of storage units, such that none of the set of encoded data slices is stored in a same storage unit of the updated plurality of storage units used previously to store a same encoded data slice of the set of encoded data slices in the original plurality of storage units.