DISTRIBUTED STORAGE NETWORK INCLUDING MEMORY DIVERSITY

• US 20110078343A1
• Filed: 05/11/2010
• Published: 03/31/2011
• Est. Priority Date: 09/29/2009
• Status: Abandoned Application

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: RESCH, JASON K., GLADWIN, S. CHRISTOPHER
• Application Number:
  US12/777,850
• Publication Number:
  US 20110078343A1
• Time in Patent Office: Days
• US Class Current: 710/33
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 2211/1011 Clustered RAID, i.e. clustered or de-clustered RAID where data and parity are spread over more disks than blocks in a parity group
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/231 Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/2405 Monitoring of the internal components or processes of the server, e.g. server load allocation of resources in multiprogramming arrangements G06F9/50; performance measurement of computer activity G06F11/34
• CPC Class Codes: H04N 21/442 Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00

Abstract

A distributed storage processing unit can generate data slices and determine metadata for each of the data slices. The metadata includes information that can be used to determine storage diversity preferences, which can include requirements that data slices generated from a common data segment each be stored in memories of the same (or different) type and model, memories with the same (or different) failure rates, memories having fast read (or write) characteristics, and so on. Decisions about which memory units to use for storing data slices can be made based on how closely the characteristics of the memories match the storage diversity preferences. The decision can be made at a distributed storage processing unit that generates the data slices, at a distributed storage unit receiving the data slices for storage, or elsewhere.

Claims

1. A method comprising:

generating a plurality of data slices in a distributed storage processing unit;

determining metadata associated with the plurality of data slices, the metadata including information that can be used to determine storage diversity preferences;

determining characteristics of a plurality of storage units included in a distributed storage network;

selecting particular storage units of the plurality of storage units based on a favorable match between the storage diversity preferences and the characteristics of the particular storage units; and

delivering the plurality of data slices to the particular storage units.

2. The method of claim 1, wherein selecting particular storage units further comprising:

identifying candidate storage units having characteristics that favorably match different storage diversity preferences;

sorting the candidate storage units based on the different storage diversity preferences; and

selecting particular candidate storage units based on a favorable match of the different storage diversity preferences.

3. The method of claim 1, wherein the storage diversity preferences include similarity and difference requirements.

4. The method of claim 1, wherein the storage diversity preferences include a requirement that a stripe used to store data slices associated with a common data segment include no more than a threshold number of memories of the same model.

5. The method of claim 1, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units selected to optimize frequent access of stored data.

6. The method of claim 1, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units selected to optimize failure rates.

7. A method comprising:

receiving a data slice to be stored in one of a plurality of memories at a dispersed storage unit;

obtaining metadata associated with the plurality data slice, the metadata including an indication of storage diversity preferences;

determining characteristics of the plurality of memories;

selecting particular memories of the plurality of memories based on a favorable match between the storage diversity preferences and the characteristics of the particular memories; and

storing the data slices to the particular memory.

8. The method of claim 7, wherein selecting particular memories further comprising:

identifying candidate memories having characteristics that favorably match storage different diversity preferences;

sorting the candidate memories based on the storage different diversity preferences; and

selecting particular candidate memories based on a favorable match of the storage different diversity preferences.

9. The method of claim 7, wherein the storage diversity preferences include similarity and difference requirements.

10. The method of claim 7, wherein the metadata includes an indication that the plurality of data slices should be stored in memories optimized for cost.

11. The method of claim 7, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units optimized for capacity.

12. A dispersed storage processing unit comprising:

a processor to generate a plurality of data slices, each of the plurality of data slices including metadata;

the processor further to select candidate storage units based on a correlation between the metadata and characteristics of the candidate storage units; and

a communications interface to deliver the plurality of data slices to the candidate storage units.

13. The dispersed storage processing unit of claim 12, wherein the processor is further to:

identify a plurality of candidate storage units having a plurality of characteristics;

sort the plurality of candidate storage units based on the plurality of characteristics; and

select particular candidate storage units based on a favorable correlation of the metadata and the plurality of characteristics of the plurality of candidate storage units.

14. The dispersed storage processing unit of claim 12, wherein the metadata includes information used to determine storage unit similarity and difference requirements.

15. The dispersed storage processing unit of claim 12, wherein the dispersed storage processing unit ensures that a read threshold number of data slices are delivered to storage units employing different types of memories.

16. The dispersed storage processing unit of claim 12, wherein the metadata includes an indication that the plurality of data slices should be stored in storage units optimized for location diversity.

17. A dispersed storage unit comprising:

a communications interface to receive a data slice for storage, the data slice including metadata;

a plurality of memory units;

a processor to select at least one of the memory units to store the data slice, the selection based on a correlation between the metadata and characteristics of the plurality of memory units;

the processor further to direct the data slices to the at least of the memory units to be stored.

18. The dispersed storage unit of claim 17, wherein the processor is further to:

identify a plurality of candidate memory units having a plurality of characteristics;

sort the plurality of candidate memory units based on the plurality of characteristics; and

select particular candidate memory units based on a favorable correlation of the metadata and the plurality of characteristics of the plurality of candidate memory units.

19. The dispersed storage unit of claim 17, wherein the metadata includes information used to determine memory unit similarity and difference requirements.

20. The dispersed storage unit of claim 17, wherein the metadata includes an indication that the data slice is to be stored in a memory unit selected to meet access speed requirements.

21. The dispersed storage unit of claim 17, wherein the metadata includes an indication that the data slice is to be stored in a memory unit selected based on a reliability requirement.