Manipulating a distributed agreement protocol to identify a desired set of storage units

• US 9,934,092 B2
• Filed: 07/12/2016
• Issued: 04/03/2018
• Est. Priority Date: 07/12/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ober, Brian F., Resch, Jason K.
• Primary Examiner(s): Bansal, Gurtej
• Application Number:
  US15/208,505
• Publication Number:
  US 20180018225A1
• Time in Patent Office: 630 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method includes obtaining, by a computing device, a plurality of sets of encoded data slices for storage in memory of a dispersed storage network (DSN). The method further includes identifying, by the computing device, a desired set of storage units within pools of storage units for storing the plurality of sets of encoded data slices. The method further includes generating, by the computing device, a specific source name based on the desired set of storage units and a distributed agreement protocol (DAP). The method further includes generating, by the computing device, a plurality of sets of slices names that includes the specific source name. The method further includes sending, by the computing device, a plurality of sets of write requests to the desired set of storage units regarding the plurality of sets of encoded data slices and in accordance with the plurality of sets of slice names.

Claims

1. A method comprises:

obtaining, by a computing device of a dispersed storage network (DSN), a plurality of sets of encoded data slices for storage in memory of the DSN, wherein the memory of the DSN includes a plurality of pools of storage units located throughout a geographic area, wherein a data object is divided into a plurality of data segments, and wherein the plurality of data segments is dispersed storage error encoded into the plurality of sets of encoded data slices;

identifying, by the computing device, a desired set of storage units within the plurality of pools of storage units for storing the plurality of sets of encoded data slices;

generating, by the computing device, a specific source name based on the desired set of storage units and a distributed agreement protocol (DAP), wherein the DAP identifies a set storage units of the plurality of pools of storage units based on a slice identifier and a plurality of storage mapping coefficients and wherein, when a device within the DSN executes the DAP, the device utilizes the specific source name as the slice identifier to identify the desired set of storage units;

generating, by the computing device, a plurality of sets of slices names for the plurality of sets of encoded data slices, wherein the plurality of sets of slices names include the specific source name; and

sending, by the computing device, a plurality of sets of write requests to the desired set of storage units regarding the plurality of sets of encoded data slices and in accordance with the plurality of sets of slice names.

2. The method of claim 1, wherein the generating the specific source name comprises:

generating a specific data object identifier; and

combining the specific data object identifier with one or more of a vault identifier and revision level information to produce the specific source name.

3. The method of claim 1, wherein generating a slice name of the plurality of sets of slices names comprises:

combining the specific source name with a pillar number and a data segment number to produce the slice name.

4. The method of claim 1, where the DAP comprises:

a plurality of functions operable to generate one or more unique scores based on a plurality of storage pool coefficients and one or more slice identifiers corresponding to the plurality of sets of encoded data slices; and

a ranking function that processes the one or more unique scores to identify a selected storage pool to store the plurality of sets of encoded data slices.

5. The method of claim 1 further comprises:

determining, by at least some storage units of the plurality of pools of storage units, a DAP imbalance resulting from computing devices creating source names having specific data object identifiers instead of randomly generating data object identifiers;

adjusting, by the at least some storage units, one or more storage mapping coefficients of the plurality of storage mapping coefficients to produce an adjusted plurality of storage mapping coefficients;

executing, by the at least some storage units, the DAP using the specific source name and the adjusted plurality of storage mapping coefficients to identify one or more encoded data slices of the plurality of sets of encoded data slices to be transferred to one or more other storage units within the plurality of pools of storage units; and

transferring, by one or more storage units of the desired set of storage units, the one or more encoded data slices to the one or more other storage units.

6. The method of claim 1, wherein the identifying the desired set of storage units comprises one or more of:

identifying a set of storage units within the plurality of pools of storage units having a desired writing speed with respect to the computing device;

identifying the set of storage units within the plurality of pools of storage units having a desired efficiency with respect to the computing device; and

identifying the set of storage units within the plurality of pools of storage units having a desired reliability with respect to the computing device.

7. The method of claim 6, wherein the identifying the desired set of storage units comprises one or more of:

executing a lookup;

initiating a query;

receiving a query response;

accessing a historical record;

accessing a table; and

receiving a desired set of storage units list.

8. A computer readable memory device comprises:

a first memory element that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to;

obtain a plurality of sets of encoded data slices for storage in memory of the DSN, wherein the memory of the DSN includes a plurality of pools of storage units located throughout a geographic area, wherein a data object is divided into a plurality of data segments, and wherein the plurality of data segments is dispersed storage error encoded into the plurality of sets of encoded data slices; and

identify a desired set of storage units within the plurality of pools of storage units for storing the plurality of sets of encoded data slices;

a second memory element that stores operational instructions that, when executed by the computing device, causes the computing device to;

generate a specific source name based on the desired set of storage units and a distributed agreement protocol (DAP), wherein the DAP identifies a set storage units of the plurality of pools of storage units based on a slice identifier and a plurality of storage mapping coefficients and wherein, when a device within the DSN executes the DAP, the device utilizes the specific source name as the slice identifier to identify the desired set of storage units; and

generate a plurality of sets of slices names for the plurality of sets of encoded data slices, wherein the plurality of sets of slices names include the specific source name; and

a third memory element that stores operational instructions that, when executed by the computing device, causes the computing device to;

send a plurality of sets of write requests to the desired set of storage units regarding the plurality of sets of encoded data slices and in accordance with the plurality of sets of slice names.

9. The computer readable memory device of claim 8, wherein the second memory element further stores operational instructions that, when executed by the computing device, causes the computing device to generate the specific source name by:

generating a specific data object identifier; and

combining the specific data object identifier with one or more of a vault identifier and revision level information to produce the specific source name.

10. The computer readable memory device of claim 8, wherein the second memory element further stores operational instructions that, when executed by the computing device, causes the computing device to generate a slice name of the plurality of sets of slices names by:

combining the specific source name with a pillar number and a data segment number to produce the slice name.

11. The computer readable memory device of claim 8, the second memory element that stores operational instructions that, when executed by the computing device, causes the computing device to utilize the DAP by performing:

a plurality of functions operable to generate one or more unique scores based on a plurality of storage pool coefficients and one or more slice identifiers corresponding to the plurality of sets of encoded data slices; and

a ranking function that processes the one or more unique scores to identify a selected storage pool to store the plurality of sets of encoded data slices.

12. The computer readable memory device of claim 8 further comprises:

a fourth memory element that stores operational instructions that, when executed by each of at least some storage units of the plurality of pools of storage units, causes the at least some storage units to;

determine a DAP imbalance resulting from computing devices creating source names having specific data object identifiers instead of randomly generating data object identifiers;

adjust one or more storage mapping coefficients of the plurality of storage mapping coefficients to produce an adjusted plurality of storage mapping coefficients;

a fifth memory element that stores operational instructions that, when executed by the at least some storage units, causes the at least some storage units to;

execute the DAP using the specific source name and the adjusted plurality of storage mapping coefficients to identify one or more encoded data slices of the plurality of sets of encoded data slices to be transferred to one or more other storage units within the plurality of pools of storage units; and

transfer, by one or more storage units of the desired set of storage units, the one or more encoded data slices to the one or more other storage units of the at least some storage units.

13. The computer readable memory device of claim 8, wherein the first memory element further stores operational instructions that, when executed by the computing device, causes the computing device to identify the desired set of storage units by one or more of:

identifying a set of storage units within the plurality of pools of storage units having a desired writing speed with respect to the computing device;

identifying the set of storage units within the plurality of pools of storage units having a desired efficiency with respect to the computing device; and

identifying the set of storage units within the plurality of pools of storage units having a desired reliability with respect to the computing device.

14. The computer readable memory device of claim 13, wherein the first memory element further stores operational instructions that, when executed by the computing device, causes the computing device to identify the desired set of storage units by one or more of:

executing a lookup;

initiating a query;

receiving a query response;

accessing a historical record;

accessing a table; and

receiving a desired set of storage units list.

15. A computing device of a dispersed storage network (DSN), wherein the computing device comprises:

interface;

memory; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

obtain a plurality of sets of encoded data slices for storage in memory of the DSN, wherein the memory of the DSN includes a plurality of pools of storage units located throughout a geographic area, wherein a data object is divided into a plurality of data segments, and wherein the plurality of data segments is dispersed storage error encoded into the plurality of sets of encoded data slices;

identify a desired set of storage units within the plurality of pools of storage units for storing the plurality of sets of encoded data slices;

generate a specific source name based on the desired set of storage units and a distributed agreement protocol (DAP), wherein the DAP identifies a set storage units of the plurality of pools of storage units based on a slice identifier and a plurality of storage mapping coefficients and wherein, when a device within the DSN executes the DAP, the device utilizes the specific source name as the slice identifier to identify the desired set of storage units;

generate a plurality of sets of slices names for the plurality of sets of encoded data slices, wherein the plurality of sets of slices names include the specific source name; and

send a plurality of sets of write requests to the desired set of storage units regarding the plurality of sets of encoded data slices and in accordance with the plurality of sets of slice names.

16. The computing device of claim 15, wherein the processing module is further operable to generate the specific source name by:

generating a specific data object identifier; and

combining the specific data object identifier with one or more of a vault identifier and revision level information to produce the specific source name.

17. The computing device of claim 15, wherein the processing module is further operable to generate a slice name of the plurality of sets of slices names by:

combining the specific source name with a pillar number and a data segment number to produce the slice name.

18. The computing device of claim 15, wherein the processing module is further operable to utilize the DAP by performing:

a plurality of functions operable to generate one or more unique scores based on a plurality of storage pool coefficients and one or more slice identifiers corresponding to the plurality of sets of encoded data slices; and

a ranking function that processes the one or more unique scores to identify a selected storage pool to store the plurality of sets of encoded data slices.

19. The computing device of claim 15, wherein the processing module is further operable to identify the desired set of storage units by one or more of:

identifying a set of storage units within the plurality of pools of storage units having a desired writing speed with respect to the computing device;

identifying the set of storage units within the plurality of pools of storage units having a desired efficiency with respect to the computing device; and

identifying the set of storage units within the plurality of pools of storage units having a desired reliability with respect to the computing device.

20. The computing device of claim 19, wherein the processing module is further operable to identify the desired set of storage units by one or more of:

executing a lookup;

initiating a query;

receiving a query response;

accessing a historical record;

accessing a table; and

receiving a desired set of storage units list.