Dispersed storage unit solicitation method and apparatus

• US 9,311,185 B2
• Filed: 06/02/2014
• Issued: 04/12/2016
• Est. Priority Date: 10/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US14/294,131
• Publication Number:
  US 20140281817A1
• Time in Patent Office: 680 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 13/1668 Details of memory controller
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/15 Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes H03M13/17 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/23 using convolutional codes, e.g. unit memory codes
• CPC Class Codes: H04L 41/06 Management of faults, events, alarms or notifications
• CPC Class Codes: H04L 41/40 using virtualisation of network functions or resources, e.g. SDN or NFV entities
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by one or more computing devices obtaining data for storage in a storage as service network environment. The method continues by determining storage preferences regarding the data. The method continues by determining a set of storage units based on the storage preferences and sending a solicitation request to the set of storage units. When at least a minimum number of favorable solicitation responses have been received within a time period, the method continues by determining a dispersed storage error encoding function based on the favorable solicitation responses, the storage preferences, and available encoding schemes. The method continues by encoding the data based on the selected dispersed storage error encoding function to produce a plurality of sets of encoded data slices. The method continues by outputting the sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses.

Claims

1. A method for execution by one or more computing devices regarding storage of data, the method comprises:

obtaining data for storage in a storage as service network environment, wherein the storage as service network environment includes a plurality of storage units that are hosted by one or more storage as service entities;

determining storage preferences regarding the data based on at least one of the data and a source of the data;

determining a set of the plurality of the storage units from storage units within one or more dispersed storage network (DSN) memories based on the storage preferences;

sending a solicitation request to the set of the plurality of the storage units, wherein the solicitation request includes a representation of the storage preferences;

when at least a minimum number of favorable solicitation responses have been received within a time period, determining a dispersed storage error encoding function based on the favorable solicitation responses, the storage preferences, and available encoding schemes;

encoding the data based on the determined dispersed storage error encoding function to produce a plurality of sets of encoded data slices; and

outputting for storage the plurality of sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses.

2. The method of claim 1, wherein the obtaining the data for storage in the storage as service network environment comprises at least one of:

retrieving the data from memory;

receiving the data from the source; and

creating the data.

3. The method of claim 1, wherein the determining the storage preferences of the data based on the data comprises at least one of:

a data type of the data;

a data size of the data;

a data application used to create the data; and

usage of the data.

4. The method of claim 1, wherein the determining the storage preferences of the data based on the source of the data comprises at least one of:

user identifier associated with the data;

geographic location requirements for physical location of storage units in the set of the plurality of the storage units;

user device identifier associated with the data;

a reliability preference; and

a retrieval preference.

5. The method of claim 1, wherein the available encoding schemes comprises:

a first encoding scheme that includes at least one of;

a first segmenting protocol;

a first pre-slice data manipulation function;

a first forward error correction encoding function;

a first slicing pillar width;

a first post-slice data manipulation function;

a first write threshold; and

a first read threshold; and

a second encoding scheme that includes at least one of;

a second segmenting protocol;

a second pre-slice data manipulation function;

a second forward error correction encoding function;

a second slicing pillar width;

a second post-slice data manipulation function;

a second write threshold; and

a second read threshold.

6. The method of claim 1, wherein the determining the dispersed storage error encoding function further comprises:

determining to locally store one or more encoded data slices of the plurality of sets of encoded data slices.

7. The method of claim 6, wherein the determining the dispersed storage error encoding function further comprises:

receiving “

y”

number of favorable solicitation responses;

determining to locally store “

f”

number of encoded data slices per set of the plurality of sets of encoded data slices; and

determining one of the available encoding schemes based on a slicing pillar width number being less than or equal to the “

y”

number plus the “

f”

number and based on a decode threshold number being of a desired error coding redundancy level.

8. The method of claim 1, wherein the determining the dispersed storage error encoding function further comprises:

receiving “

y”

number of favorable solicitation responses; and

determining one of the available encoding schemes based on a slicing pillar width number being less than or equal to the “

y”

number and based on a decode threshold number being of a desired error coding redundancy level.

9. The method of claim 1, wherein the outputting for storage the plurality of sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses further comprises:

storing a file path for the data, wherein the file path includes memory addresses of the storage units and a file name of the data.

10. A computing device comprises:

an interface;

a memory; and

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

obtain, via the interface, data for storage in a storage as service network environment, wherein the storage as service network environment includes a plurality of storage units that are hosted by one or more storage as service entities;

determine storage preferences regarding the data based on at least one of the data and a source of the data;

determine a set of the plurality of the storage units from storage units within one or more dispersed storage network (DSN) memories based on the storage preferences;

send, via the interface, a solicitation request to the set of the plurality of the storage units, wherein the solicitation request includes a representation of the storage preferences;

when at least a minimum number of favorable solicitation responses have been received within a time period, determine a dispersed storage error encoding function based on the favorable solicitation responses, the storage preferences, and available encoding schemes;

encode the data based on the determined dispersed storage error encoding function to produce a plurality of sets of encoded data slices; and

output for storage, via the interface, the plurality of sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses.

11. The computing device of claim 10, wherein the processing module further functions to obtain the data for storage in the storage as service network environment by at least one of:

retrieving the data from memory;

receiving the data from the source; and

creating the data.

12. The computing device of claim 10, wherein the processing module further functions to determine the storage preferences of the data based on the data by at least one of:

a data type of the data;

a data size of the data;

a data application used to create the data; and

usage of the data.

13. The computing device of claim 10, wherein the processing module further functions to determine the storage preferences of the data based on the source of the data by at least one of:

user identifier associated with the data;

geographic location requirements for physical location of storage units in the set of the plurality of the storage units;

user device identifier associated with the data;

a reliability preference; and

a retrieval preference.

14. The computing device of claim 10, wherein the available encoding schemes comprises:

a first encoding scheme that includes at least one of;

a segmenting protocol;

a pre-slice data manipulation function;

a forward error correction encoding function;

a slicing pillar width;

a post-slice data manipulation function;

a write threshold; and

a read threshold; and

a second encoding scheme that includes at least one of;

a second segmenting protocol;

a second pre-slice data manipulation function;

a second forward error correction encoding function;

a second slicing pillar width;

a second post-slice data manipulation function;

a second write threshold; and

a second read threshold.

15. The computing device of claim 10, wherein the processing module further functions to determine the dispersed storage error encoding function by:

determining to locally store one or more encoded data slices of the plurality of sets of encoded data slices.

16. The computing device of claim 15, wherein the processing module further functions to determine the dispersed storage error encoding function by:

receiving “

y”

number of favorable solicitation responses;

determining to locally store “

f”

number of encoded data slices per set of the plurality of sets of encoded data slices; and

determining one of the available encoding schemes based on a slicing pillar width number being less than or equal to the “

y”

number plus the “

f”

number and based on a decode threshold number being of a desired error coding redundancy level.

17. The computing device of claim 10, wherein the processing module further functions to determine the dispersed storage error encoding function by:

receiving “

y”

number of favorable solicitation responses; and

determining one of the available encoding schemes based on a slicing pillar width number being less than or equal to the “

y”

number and based on a decode threshold number being of a desired error coding redundancy level.

18. The computing device of claim 10, wherein the processing module further functions to output for storage, via the interface, the plurality of sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses by:

storing a file path for the data, wherein the file path includes memory addresses of the storage units and a file name of the data.

19. A computer readable storage memory comprises:

a first storage section that stores operational instructions that, when executed by one or more processing modules, causes the one or more processing modules to;

obtain data for storage in a storage as service network environment, wherein the storage as service network environment includes a plurality of storage units that are hosted by one or more storage as service entities;

a second storage section that stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to;

determine storage preferences regarding the data based on at least one of the data and a source of the data;

a third storage section that stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to;

determine a set of the plurality of storage units from storage units within one or more dispersed storage network (DSN) memories based on the storage preferences;

send a solicitation request to the set of the plurality of storage units, wherein the solicitation request includes a representation of the storage preferences;

when at least a minimum number of favorable solicitation responses have been received within a time period, determine a dispersed storage error encoding function based on the favorable solicitation responses, the storage preferences, and available encoding schemes;

encode the data based on the determined dispersed storage error encoding function to produce a plurality of sets of encoded data slices; and

output for storage the plurality of sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses.

20. The computer readable storage memory of claim 19, wherein the first storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to obtain the data for storage in the storage as service network environment by at least one of:

retrieving the data from memory;

receiving the data from the source; and

creating the data.

21. The computer readable storage memory of claim 19, wherein the second storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to determine the storage preferences of the data based on the data by at least one of:

a data type of the data;

a data size of the data;

a data application used to create the data; and

usage of the data.

22. The computer readable storage memory of claim 19, wherein the second storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to determine the storage preferences of the data based on the source of the data by at least one of:

user identifier associated with the data;

geographic location requirements for physical location of storage units in the set of the plurality of the storage units;

user device identifier associated with the data;

a reliability preference; and

a retrieval preference.

23. The computer readable storage memory of claim 19, wherein the available encoding schemes comprises:

a first encoding scheme that includes at least one of;

a first segmenting protocol;

a first pre-slice data manipulation function;

a first forward error correction encoding function;

a first slicing pillar width;

a first post-slice data manipulation function;

a first write threshold; and

a first read threshold; and

a second encoding scheme that includes at least one of;

a second segmenting protocol;

a second pre-slice data manipulation function;

a second forward error correction encoding function;

a second slicing pillar width;

a second post-slice data manipulation function;

a second write threshold; and

a second read threshold.

24. The computer readable storage memory of claim 19, wherein the third storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to determine the dispersed storage error encoding function by:

determining to locally store one or more encoded data slices of the plurality of sets of encoded data slices.

25. The computer readable storage memory of claim 24, wherein the third storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to determine the dispersed storage error encoding function by:

receiving “

y”

number of favorable solicitation responses;

determining to locally store “

f”

number of encoded data slices per set of the plurality of sets of encoded data slices; and

determining one of the available encoding schemes based on a slicing pillar width number being less than or equal to the “

y”

number plus the “

f”

number and based on a decode threshold number being of a desired error coding redundancy level.

26. The computer readable storage memory of claim 19, wherein the third storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to determine the dispersed storage error encoding function by:

receiving “

y”

number of favorable solicitation responses; and

determining one of the available encoding schemes based on a slicing pillar width number being less than or equal to the “

y”

number and based on a decode threshold number being of a desired error coding redundancy level.

27. The computer readable storage memory of claim 19, wherein the third storage section further stores operational instructions that, when executed by the one or more processing modules, causes the one or more processing modules to output for storage the plurality of sets of encoded data slices to at least some of the storage units providing the favorable solicitation responses by:

storing a file path for the data, wherein the file path includes memory addresses of the storage units a file name of the data.