Reallocation in a dispersed storage network (DSN)

• US 9,952,930 B2
• Filed: 10/21/2016
• Issued: 04/24/2018
• Est. Priority Date: 10/30/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Abhijeet, Kumar, Motwani, Manish, Resch, Jason K., Wozniak, Ethan S.
• Primary Examiner(s): Knapp, Justin R
• Application Number:
  US15/331,484
• Publication Number:
  US 20170123910A1
• Time in Patent Office: 550 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 43/0864 Round trip delays
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. When a is to DSN undergo a change from a first system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP) to a second system configuration of the DAP (e.g., such as based on addition, and/or removal of storage unit(s) (SU(s)) within the DSN or reallocation of data within the DSN, etc.), a computing device identifies a DAP transition mapping between the first system configuration of the DAP to the second system configuration of the DAP. Then, the computing device directs SU(s) to operate based on the DAP transition mapping during the transition.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

detect a change within the DSN from a first system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP) to a second system configuration of the DAP based on at least one of addition or removal of at least one storage unit (SU) within a plurality of storage units (SUs) within the DSN or reallocation of data within the DSN, wherein the first system configuration of the DAP and the second system configuration of the DAP respectively provide for deterministic calculation of locations of encoded data slice (EDS) sets that correspond respectively to a plurality of data segments of a data object that are distributedly stored among the plurality of storage units SUs within the DSN, wherein the data object is segmented into the plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs of the EDS sets;

identify a DAP transition mapping between the first system configuration of the DAP to the second system configuration of the DAP;

direct at least one SU to service at least one of a first read request or a first write request from a first other computing device for at least one EDS stored within the at least one SU based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP; and

direct the at least one SU to service at least one of a second read request or a second write request from at least one of the first other computing device or a second other computing device for the at least one EDS stored within the at least one SU based on the second system configuration of the DAP after completion of the transition from the first system configuration of the DAP to the second system configuration of the DAP.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

direct a subset of SUs within the plurality of SUs that store the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a first other read request or a first other write request from the first other computing device for at least one EDS of the EDS sets based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP when performed on a data object-by-data object basis; and

direct the subset of SUs within the plurality of SUs that store the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a second other read request or a second other write request from at least one of the first other computing device or the second other computing device for the at least one EDS of the EDS sets based on the second system configuration of the DAP after completion of the transition of the subset of SUs within the plurality of SUs from the first system configuration of the DAP to the second system configuration of the DAP.

3. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

direct a subset of SUs within the plurality of SUs that store an EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a first other read request or a first other write request from the first other computing device for at least one EDS of the EDS set of the EDS sets based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP when performed on an EDS set-by-EDS set basis; and

direct the subset of SUs within the plurality of SUs that store the EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a second other read request or a second other write request from at least one of the first other computing device or the second other computing device for the at least one EDS of the EDS set of the EDS sets based on the second system configuration of the DAP after completion of the transition of the subset of SUs within the plurality of SUs from the first system configuration of the DAP to the second system configuration of the DAP.

4. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

direct a subset of SUs within the plurality of SUs of a vault that store at least some of an EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a first other read request or a first other write request from the first other computing device for at least one EDS of the EDS set of the EDS sets based on the DAP transition mapping during transition of the subset of SUs within the plurality of SUs of the vault from the first system configuration of the DAP to the second system configuration of the DAP when performed on a vault-by-vault basis; and

direct the subset of SUs within the plurality of SUs of the vault that store the at least some of the EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a second other read request or a second other write request from at least one of the first other computing device or the second other computing device for the at least some of the EDS set of the EDS sets based on the second system configuration of the DAP after completion of the transition from the first system configuration of the DAP to the second system configuration of the DAP.

5. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

direct the plurality of SUs to service any other read request or any other write request from at least one of the first other computing device or the second other computing device for any EDS within the EDS sets based on the second system configuration of the DAP after completion of transition of the plurality of SUs from the first system configuration of the DAP to the second system configuration of the DAP.

6. The computing device of claim 1, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

7. The computing device of claim 1 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

detect a change within the DSN from a first system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP) to a second system configuration of the DAP based on at least one of addition or removal of at least one storage unit (SU) within a plurality of storage units (SUs) within the DSN or reallocation of data within the DSN, wherein the first system configuration of the DAP and the second system configuration of the DAP respectively provide for deterministic calculation of locations of encoded data slice (EDS) sets that correspond respectively to a plurality of data segments of a data object that are distributedly stored among the plurality of storage units SUs within the DSN, wherein the data object is segmented into the plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs of the EDS sets;

identify a DAP transition mapping between the first system configuration of the DAP to the second system configuration of the DAP;

direct the plurality of SUs to service at least one of a first read request or a first write request from a first other computing device for at least one EDS within the EDS sets stored within the plurality of SUs based on at least one of a data object-by-data object basis, an EDS set-by-EDS set basis, or vault-by-vault basis and based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP; and

direct the plurality of SUs to service at least one of a second read request or a second write request from at least one of the first other computing device or a second other computing device for the at least one EDS within the EDS sets stored within the plurality of SUs based on the second system configuration of the DAP after completion of the transition from the first system configuration of the DAP to the second system configuration of the DAP.

10. The computing device of claim 9, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

direct a first subset of the plurality of SUs to service the at least one of the first read request or the first write request from the first other computing device for the at least one EDS within the EDS sets stored within the plurality of SUs based on the data object-by-data object basis and based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP; and

direct a second subset of the plurality of SUs to service at least one of another read request or another write request from at least one of the first other computing device or the second other computing device for at least one of the EDS or at least one other EDS within the EDS sets stored within the plurality of SUs based on the EDS set-by-EDS set basis or the vault-by-vault basis and based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP.

11. The computing device of claim 9, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

12. The computing device of claim 9 further comprising:

a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

detecting, via an interface of the computing device configured to interface and communicate with a dispersed storage network (DSN), a change within the DSN from a first system configuration of a Decentralized, or Distributed, Agreement Protocol (DAP) to a second system configuration of the DAP based on at least one of addition or removal of at least one storage unit (SU) within a plurality of storage units (SUs) within the DSN or reallocation of data within the DSN, wherein the first system configuration of the DAP and the second system configuration of the DAP respectively provide for deterministic calculation of locations of encoded data slice (EDS) sets that correspond respectively to a plurality of data segments of a data object that are distributedly stored among the plurality of storage units SUs within the DSN, wherein the data object is segmented into the plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of EDSs of the EDS sets;

identifying a DAP transition mapping between the first system configuration of the DAP to the second system configuration of the DAP;

directing, via the interface of the computing device, at least one SU to service at least one of a first read request or a first write request from a first other computing device for at least one EDS stored within the at least one SU based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP; and

directing, via the interface of the computing device, the at least one SU to service at least one of a second read request or a second write request from at least one of the first other computing device or a second other computing device for the at least one EDS stored within the at least one SU based on the second system configuration of the DAP after completion of the transition from the first system configuration of the DAP to the second system configuration of the DAP.

15. The method of claim 14 further comprising:

directing a subset of SUs within the plurality of SUs that store the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a first other read request or a first other write request from the first other computing device for at least one EDS of the EDS sets based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP when performed on a data object-by-data object basis; and

directing the subset of SUs within the plurality of SUs that store the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a second other read request or a second other write request from at least one of the first other computing device or the second other computing device for the at least one EDS of the EDS sets based on the second system configuration of the DAP after completion of the transition of the subset of SUs within the plurality of SUs from the first system configuration of the DAP to the second system configuration of the DAP.

16. The method of claim 14 further comprising:

directing a subset of SUs within the plurality of SUs that store an EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a first other read request or a first other write request from the first other computing device for at least one EDS of the EDS set of the EDS sets based on the DAP transition mapping during transition from the first system configuration of the DAP to the second system configuration of the DAP when performed on an EDS set-by-EDS set basis; and

directing the subset of SUs within the plurality of SUs that store the EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a second other read request or a second other write request from at least one of the first other computing device or the second other computing device for the at least one EDS of the EDS set of the EDS sets based on the second system configuration of the DAP after completion of the transition of the subset of SUs within the plurality of SUs from the first system configuration of the DAP to the second system configuration of the DAP.

17. The method of claim 14 further comprising:

directing a subset of SUs within the plurality of SUs of a vault that store at least some of an EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a first other read request or a first other write request from the first other computing device for at least one EDS of the EDS set of the EDS sets based on the DAP transition mapping during transition of the subset of SUs within the plurality of SUs of the vault from the first system configuration of the DAP to the second system configuration of the DAP when performed on a vault-by-vault basis; and

directing the subset of SUs within the plurality of SUs of the vault that store the at least some of the EDS set of the EDS sets that correspond respectively to the plurality of data segments of the data object to service at least one of a second other read request or a second other write request from at least one of the first other computing device or the second other computing device for the at least some of the EDS set of the EDS sets based on the second system configuration of the DAP after completion of the transition from the first system configuration of the DAP to the second system configuration of the DAP.

18. The method of claim 14 further comprising:

directing the plurality of SUs to service any other read request or any other write request from at least one of the first other computing device or the second other computing device for any EDS within the EDS sets based on the second system configuration of the DAP after completion of transition of the plurality of SUs from the first system configuration of the DAP to the second system configuration of the DAP.

19. The method of claim 14, wherein the computing device is a SU of the plurality of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).