Configuring storage resources of a dispersed storage network

• US 9,594,639 B2
• Filed: 10/29/2014
• Issued: 03/14/2017
• Est. Priority Date: 01/06/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Khadiwala, Ravi, Leggette, Wesley, Baptist, Andrew, Dhuse, Greg, Volvovski, Ilya, Resch, Jason K., Motwani, Manish
• Primary Examiner(s): LE, DIEU MINH T
• Application Number:
  US14/527,139
• Publication Number:
  US 20150193309A1
• Time in Patent Office: 867 Days
• Field of Search: 714/6.11, 714/6.12, 714/6.13, 714/6.2, 714/4.1, 714/4.11, 714/4.3, 714/4.12
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/353 Adaptation to the channel
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module of a dispersed storage network (DSN) ascertaining a decode threshold value for dispersed storage error encoding data for storage in storage units of the DSN. The method continues with the processing module determining a total width value for the dispersed storage error encoding based on the decode threshold value, a number of selected sites within the DSN, and a number of selected storage units of the selected sites. The method continues with the processing module determining logical storage slots within the selected storage units based on the total width value, the number of selected, and the number of selected storage units. The method continues with the processing module writing a set of encoded data slices to a total width value of the logical storage slots within at least some of the selected storage units of the selected sites based on a slice-to-slot mapping.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

ascertaining a decode threshold value for dispersed storage error encoding data for storage in storage units of the DSN, wherein the decode threshold value corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a data segment of the data, wherein the data segment was dispersed storage error encoded into the set of encoded data slices;

determining a total width value for the dispersed storage error encoding based on the decode threshold value, a number of selected sites within the DSN, and a number of selected storage units of the selected sites, wherein the total width value is greater than the decode threshold value and corresponds to a number of encoded data slices in the set of encoded data slices;

determining logical storage slots within the selected storage units of the selected sites based on the total width value, the number of selected sites within the DSN, and the number of selected storage units of the selected sites; and

writing the set of encoded data slices to a total width value of the logical storage slots within at least some of the selected storage units of the selected sites based on a slice-to-slot mapping, wherein the slice-to-slot mapping allows for a site failure number of the selected sites to fail and still be able to retrieve the decode threshold value of encoded data slices from one or more of the selected storage units of remaining selected sites of the selected sites.

2. The method of claim 1 further comprises:

ascertaining the decode threshold value on a segment-by-segment basis for the data;

or

ascertaining the decode threshold value for the data.

3. The method of claim 1, wherein the determining the total width value comprises:

generating an expansion factor based on the number of selected sites and the site failure number; and

performing a mathematical function on the decode threshold value and the expansion factor to generate the total width value.

4. The method of claim 1 further comprises:

determining available sites within the DSN; and

selecting the selected sites from the available sites.

5. The method of claim 4 further comprises:

identifying available storage units within the selected sites; and

selecting the selected storage units from the available storage units.

6. The method of claim 1, wherein the determining the logical storage slots comprises:

determining a slot number to indicate a number of slots to include in the logical storage slots, wherein the slot number is greater than the total width value.

7. The method of claim 6 further comprises:

generating an expansion factor based on the number of selected sites and the site failure number; and

performing a mathematical function on the total width value and the expansion factor to generate the slot number.

8. The method of claim 1 further comprises:

determining the slice-to-slot mapping based on the site failure number and a load balancing scheme among the at least some of the selected storage units.

9. The method of claim 1, wherein the writing the set of encoded data slices comprises:

determining which of the selected sites are available; and

when a number of available selected sites is equal to or greater than the site failure number, selecting the at least some of the selected sites from the available selected sites in accordance with the slice-to-slot mapping.

10. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

ascertain a decode threshold value for dispersed storage error encoding data for storage in storage units of the DSN, wherein the decode threshold value corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a data segment of the data, wherein the data segment was dispersed storage error encoded into the set of encoded data slices;

determine a total width value for the dispersed storage error encoding based on the decode threshold value, a number of selected sites within the DSN, and a number of selected storage units of the selected sites, wherein the total width value is greater than the decode threshold value and corresponds to a number of encoded data slices in the set of encoded data slices;

determine logical storage slots within the selected storage units of the selected sites based on the total width value, the number of selected sites within the DSN, and the number of selected storage units of the selected sites; and

write the set of encoded data slices to a total width value of the logical storage slots within at least some of the selected storage units of the selected sites based on a slice-to-slot mapping, wherein the slice-to-slot mapping allows for a site failure number of the selected sites to fail and still be able to retrieve the decode threshold value of encoded data slices from one or more of the selected storage units of remaining selected sites of the selected sites.

11. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

ascertain the decode threshold value on a segment-by-segment basis for the data;

or

ascertain the decode threshold value for the data.

12. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to determine the total width value by:

generating an expansion factor based on the number of selected sites and the site failure number; and

performing a mathematical function on the decode threshold value and the expansion factor to generate the total width value.

13. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine available sites within the DSN; and

select the selected sites from the available sites.

14. The non-transitory computer readable storage medium of claim 13 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

identify available storage units within the selected sites; and

select the selected storage units from the available storage units.

15. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to determine the logical storage slots by:

determining a slot number to indicate a number of slots to include in the logical storage slots, wherein the slot number is greater than the total width value.

16. The non-transitory computer readable storage medium of claim 15 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

generate an expansion factor based on the number of selected sites and the site failure number; and

perform a mathematical function on the total width value and the expansion factor to generate the slot number.

17. The non-transitory computer readable storage medium of claim 10 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

determine the slice-to-slot mapping based on the site failure number and a load balancing scheme among the at least some of the selected storage units.

18. The non-transitory computer readable storage medium of claim 10, wherein the one or more processing modules functions to execute the operational instructions stored by the at least one memory section to cause the one or more computing devices of the DSN to write the set of encoded data slices by:

determining which of the selected sites are available; and

when a number of available selected sites is equal to or greater than the site failure number, selecting the at least some of the selected sites from the available selected sites in accordance with the slice-to-slot mapping.

19. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

ascertain a decode threshold value for dispersed storage error encoding data for storage in storage units of the DSN, wherein the decode threshold value corresponds to a minimum number of encoded data slices of a set of encoded data slices needed to recover a data segment of the data, wherein the data segment was dispersed storage error encoded into the set of encoded data slices;

determine a total width value for the dispersed storage error encoding based on the decode threshold value, a number of selected sites within the DSN, and a number of selected storage units of the selected sites, wherein the total width value is greater than the decode threshold value and corresponds to a number of encoded data slices in the set of encoded data slices;

determine logical storage slots within the selected storage units of the selected sites based on the total width value, the number of selected sites within the DSN, and the number of selected storage units of the selected sites; and

write the set of encoded data slices to a total width value of the logical storage slots within at least some of the selected storage units of the selected sites based on a slice-to-slot mapping, wherein the slice-to-slot mapping allows for a site failure number of the selected sites to fail and still be able to retrieve the decode threshold value of encoded data slices from one or more of the selected storage units of remaining selected sites of the selected sites.

20. The computing device of claim 19, wherein the processing module further functions to:

ascertain the decode threshold value on a segment-by-segment basis for the data;

or

ascertain the decode threshold value for the data.

21. The computing device of claim 19, wherein the processing module functions to determine the total width value by:

generating an expansion factor based on the number of selected sites and the site failure number; and

performing a mathematical function on the decode threshold value and the expansion factor to generate the total width value.

22. The computing device of claim 19, wherein the processing module further functions to:

determine available sites within the DSN; and

select the selected sites from the available sites.

23. The computing device of claim 22, wherein the processing module further functions to:

identify available storage units within the selected sites; and

select the selected storage units from the available storage units.

24. The computing device of claim 19, wherein the processing module functions to determine the logical storage slots by:

determining a slot number to indicate a number of slots to include in the logical storage slots, wherein the slot number is greater than the total width value.

25. The computing device of claim 24, wherein the processing module further functions to:

generate an expansion factor based on the number of selected sites and the site failure number; and

perform a mathematical function on the total width value and the expansion factor to generate the slot number.

26. The computing device of claim 19, wherein the processing module further functions to:

determine the slice-to-slot mapping based on the site failure number and a load balancing scheme among the at least some of the selected storage units.

27. The computing device of claim 19, wherein the processing module functions to write the set of encoded data slices by:

determining which of the selected sites are available; and

when a number of available selected sites is equal to or greater than the site failure number, selecting the at least some of the selected sites from the available selected sites in accordance with the slice-to-slot mapping.