Dispersed storage unit selection

• US 8,281,182 B2
• Filed: 05/13/2010
• Issued: 10/02/2012
• Est. Priority Date: 03/12/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Gladwin, S. Christopher
• Primary Examiner(s): Manoskey, Joseph D
• Application Number:
  US12/779,658
• Publication Number:
  US 20110225466A1
• Time in Patent Office: 873 Days
• Field of Search: 714/6.1, 714/6.2, 714/6.3, 714/6.31, 714/20, 714/42, 714/54, 711/154, 711/163
• US Class Current: 714/6.2
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 12/0646 Configuration or reconfiguration
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A dispersed storage device for use within a dispersed storage network operates to select a set of dispersed storage units for storage of a data object by slicing an encoded data segment of a data object into error coded data slices, determining slice metadata for the error coded data slices, determining memory characteristics of dispersed storage units capable of storing the error coded data slices and selecting the set of dispersed storage units for storing the error coded data slices based on the slice metadata and the memory characteristics.

Claims

1. A dispersed storage device for use within a dispersed storage network, comprising:

an interface; and

a processing module operable to;

slice an encoded data segment of a data object into error coded data slices;

determine slice metadata for the error coded data slices;

determine memory characteristics of dispersed storage units capable of storing the error coded data slices;

select a set of dispersed storage units for storing the error coded data slices based on the slice metadata and the memory characteristics; and

output, via the interface, the error coded data slices to corresponding ones of the set of dispersed storage units;

wherein the processing module is further operable to determine storage requirements from the slice metadata and to compare the storage requirements to the memory characteristics to select the set of dispersed storage units.

2. The dispersed storage device of claim 1, wherein the slice metadata comprises one or more of a file type, file size, priority, security index, user group affiliation, estimated storage time and estimated time between retrievals.

3. The dispersed storage device of claim 1, wherein the memory characteristics comprise one or more of a geographical location, proximity to other dispersed storage units, memory model, memory type, total capacity, available capacity, access speed, error history, estimated mean time between failures, actual mean time between failures and hours of operation.

4. The dispersed storage device of claim 1, wherein the dispersed storage units are organized into memory systems such that each of the memory systems includes at least one of the dispersed storage units and the memory systems are geographically separated from one another.

5. The dispersed storage device of claim 4, wherein:

a number of the error coded data slices of the data segment corresponds to a number of pillars for storing the data object; and

the storage requirements comprise a first security restriction that requires the number of pillars stored in a single one of the memory systems compare favorably to a pillar threshold.

6. The dispersed storage device of claim 5, wherein the storage requirements comprise a second security restriction that requires that a number of the error coded data slices stored in a single one of the dispersed storage units compare favorably with a slice threshold.

7. The dispersed storage device of claim 1, wherein the storage requirements comprise a fast retrieval characteristic that requires a retrieval time of the error coded data slices from each dispersed storage unit within the set of dispersed storage units compare favorably to a retrieval time threshold.

8. The dispersed storage device of claim 1, wherein the storage requirements comprise a first reliability characteristic that requires different memory models for at least two of the set of dispersed storage units.

9. The dispersed storage device of claim 1, wherein the storage requirements comprise a reliability characteristic that requires each dispersed storage unit within the set of dispersed storage units to have an error history that compares favorably to an error threshold.

10. The dispersed storage device of claim 1, wherein the storage requirements comprise a capacity characteristic that requires each dispersed storage unit within the set of dispersed storage units to have a capacity substantially the same as other dispersed storage units within the set of dispersed storage units.

11. The dispersed storage device of claim 1, wherein the storage requirements comprise an availability characteristic that requires each dispersed storage unit within the set of dispersed storage units to have an available capacity that compares favorably to a threshold amount.

12. The dispersed storage device of claim 1, wherein the storage requirements comprise a diversity characteristic that requires the set of dispersed storage units to include dispersed storage units with at least one of diverse memory models, diverse memory types, diverse error histories, diverse mean times between failures or diverse hours of operation.

13. A method for selecting dispersed storage units within a dispersed storage network, comprising:

slicing a data segment of a data object into error coded data slices;

determining slice metadata for the error coded data slices;

determining memory characteristics of dispersed storage units capable of storing the error coded data slices;

selecting a set of dispersed storage units for storing the error coded data slices based on the slice metadata and the memory characteristics, wherein the selecting further comprises;

determining storage requirements from the slice metadata; and

comparing the storage requirements to the memory characteristics to select the set of dispersed storage units; and

outputting, via the interface, the error coded data slices to corresponding ones of the set of dispersed storage units.

14. The method of claim 13, wherein:

the dispersed storage units are organized into memory systems such that each of the memory systems includes at least one of the dispersed storage units and the memory systems are geographically separated from one another;

a number of the error coded data slices of the data segment corresponds to a number of pillars for storing the data object; and

the determining the storage requirements further comprises;

determining a first security restriction that requires the number of pillars stored in a single one of the memory systems compare favorably to a pillar threshold.

15. The method of claim 14, wherein the determining the storage requirements further comprises:

determining a second security restriction that requires that a number of the error coded data slices stored in a single one of the dispersed storage units compare favorably with a slice threshold.

16. The method of claim 13, wherein the determining the storage requirements further comprises:

determining a fast retrieval characteristic that requires a retrieval time of the error coded data slices from each dispersed storage unit within the set of dispersed storage units compare favorably to a retrieval time threshold.

17. The method of claim 13, wherein the determining the storage requirements comprises:

determining a reliability characteristic that requires different memory models for at least two of the set of dispersed storage units.

18. The method of claim 13, wherein the determining the storage requirements comprises:

determining a reliability characteristic that requires each dispersed storage unit within the set of dispersed storage units to have an error history that compares favorably to an error threshold.

19. The method of claim 13, wherein the determining the storage requirements further comprises:

determining a capacity characteristic that requires each dispersed storage unit within the set of dispersed storage units to have a capacity substantially the same as other dispersed storage units within the set of dispersed storage units.

20. The method of claim 13, wherein the determining the storage requirements further comprises:

determining an availability characteristic that requires each dispersed storage unit within the set of dispersed storage units to have an available capacity that compares favorably to a threshold amount.

21. The method of claim 13, wherein the determining the storage requirements further comprises:

determining a diversity characteristic that requires the set of dispersed storage units to include dispersed storage units with at least one of diverse memory models, diverse memory types, diverse error histories, diverse mean times between failures or diverse hours of operation.