Distributed storage network for storing a data object based on storage requirements

• US 8,769,035 B2
• Filed: 07/19/2010
• Issued: 07/01/2014
• Est. Priority Date: 10/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Leggette, Wesley, Gladwin, S. Christopher, Dhuse, Greg, Baptist, Andrew
• Primary Examiner(s): ZHANG, SHIRLEY X
• Application Number:
  US12/839,181
• Publication Number:
  US 20110106904A1
• Time in Patent Office: 1,443 Days
• Field of Search: None
• US Class Current: 709/211
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/20 Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
• CPC Class Codes: H04N 21/218 Source of audio or video content , e.g. local disk arrays details of retrieval in video databases G06F16/739

Abstract

A distributed storage network (DSN) includes a user device and a plurality of DSN memories, wherein each of the DSN memories includes a plurality of storage units. The user device includes at least one network interface to the plurality of DSN memories and at least one processing module that is operable to determine one of the plurality of DSN memories for storing a data object based on a comparison of one or more storage requirements of the data object and one or more DSN attributes of the plurality of DSN memories.

Claims

1. A method for storing a data object in a distributed storage network (DSN), wherein the method comprises:

receiving a store request message to store a data object, wherein the store request message includes the data object and one or more storage requirements for the data object and wherein the store request message includes a storage requirement of the one or more storage requirements having a first higher priority level and a second storage requirement of the one or more storage requirements having a second lower priority level;

determining one or more DSN attributes associated with each of a plurality of DSN memories, wherein the plurality of DSN memories includes a plurality of storage units;

selecting one of the plurality of DSN memories in response to a comparison of the one or more DSN attributes and the one or more storage requirements for the data object to produce a selected DSN memory;

partitioning the data object into a plurality of data segments and processing the plurality of data segments to generate a plurality of encoded data slices based on an error encoding dispersal function;

transmitting the plurality of encoded data slices to the plurality of storage units of the selected DSN memory; and

transmitting an address update message to a DS managing unit to update a user vault with the selected DSN memory for the data object.

2. The method of claim 1, further comprising determining the one or more storage requirements for the data object by:

determining a priority level of the data object; and

accessing a resource that includes storage requirements for the priority level of the data object.

3. The method of claim 1, further comprising:

receiving the store request message for the data object from a user device; and

accessing a local access control list (ACL) to authenticate the user device.

4. The method of claim 1, wherein selecting one of the plurality of DSN memories includes:

allocating a greater weight to a first favorable comparison between a first DSN attribute and one of the storage requirements with the first higher priority level; and

allocating a lesser weight to a second favorable comparison between a second DSN attribute and one of the storage requirements with the second lower priority level.

5. The method of claim 4, wherein the one or more DSN attributes includes one or more of the following:

cost, geographic location, geographic diversity, performance, reliability, latency, capacity, load and availability.

6. The method of claim 5, further comprising:

receiving periodic updates on the one or more DSN attributes for the plurality of DSN memories.

7. The method of claim 1, further comprising:

generating billing information in response to storing the data object in the selected DSN memory; and

transmitting the billing information to a managing unit for aggregation into one or more billing records associated with a user.

8. A user device in a distributed storage network (DSN), comprises:

at least one network interface to the distributed storage network; and

at least one processing module that is operable to;

determine to store a data object;

determine one or more storage requirements for storing the data object in one of a plurality of DSN memories in the distributed storage network, wherein a first storage requirement of the one or more storage requirements has a first higher priority level and a second storage requirement of the one or more storage requirements has a second lower priority level;

determine one or more DSN attributes associated with each of the plurality of DSN memories;

select one of the plurality of DSN memories to produce a selected DSN memory in response to a comparison of the one or more storage requirements of the data object with the one or more DSN attributes for the selected DSN memory;

select one of a plurality of DSN processing units to produce a selected DSN processing unit;

transmit a store request message to the selected DSN processing unit to generate a plurality of encoded data slices from the data object using an error encoding dispersal function and transmit the plurality of encoded data slices to the selected DSN memory; and

transmit an address update message to a DS managing unit to update a user vault with the selected DSN memory for the data object.

9. The user device of claim 8, wherein the processing module is further operable to:

determine a priority level of the data object; and

accessing a resource that includes storage requirements for the priority level of the data object.

10. The user device of claim 8, wherein the processing module is further operable to:

allocate a greater weight to a first favorable comparison between a first DSN attribute and the first storage requirement with the first higher priority level; and

allocate a lesser weight to a second favorable comparison between a second DSN attribute and the second storage requirement with the second lower priority level.

11. The user device of claim 10, wherein the one or more DSN attributes includes one or more of the following:

cost, geographic location, geographic diversity, performance, reliability, latency, capacity, load and availability.

12. A distributed storage (DS) processing unit in a distributed storage network (DSN), comprises:

at least one network interface to a plurality of DSN memories in the distributed storage network, wherein each of the plurality of DSN memories includes a plurality of storage units; and

at least one processing module that is operable to;

receive a store request message for storing a data object, wherein the store request message includes one or more storage requirements for the data object and wherein the store request message includes a storage requirement of the one or more storage requirements having a first higher priority level and a second storage requirement of the one or more storage requirements having a second lower priority level;

determine one or more DSN attributes associated with each of the plurality of DSN memories;

select one of the plurality of DSN memories to generate a selected DSN memory in response to a comparison of the one or more storage requirements for the data object with the one or more DSN attributes of the plurality of DSN memories;

partition the data object into a plurality of data segments and process the plurality of data segments to generate a plurality of encoded data slices using an error encoding dispersal function;

transmit the plurality of encoded data slices to the plurality of storage units of the selected DSN memory; and

transmit an address update message to a DS managing unit to update a user vault with the selected DSN memory for the data object.

13. The DS processing unit of claim 12, wherein the at least one processing module is further operable to:

allocate a greater weight to a first favorable comparison between a first DSN attribute and the first storage requirement with the first higher priority level; and

allocate a lesser weight to a second favorable comparison between a second DSN attribute and the second storage requirement with the second lower priority level.

14. The DS processing unit of claim 13, wherein the one or more DSN attributes includes one or more of the following:

cost, geographic location, geographic diversity, performance, reliability, latency, capacity, load and availability.

15. The DS processing unit of claim 12, wherein the at least one processing module is further operable to:

generate a billing record in response to storing the data object in the selected DSN memory; and

transmit the billing record to the DS managing unit for aggregation with into one or more billing records associated with a user.

16. The DS processing unit of claim 12, wherein the at least one processing module is further operable to:

receive a local access control list (ACL) from the DS managing unit; and

access the local access control list (ACL) to authenticate a user device identified as a source of the store request message.

17. The DS processing unit of claim 12, wherein the at least one processing module is operable to partition the data object into a plurality of data segments and process a data segment of the plurality of data segments to generate a plurality of encoded data slices using the error encoding dispersal function by:

encoding the plurality of data segments using the error encoding dispersal function to generate a plurality of encoded data segments; and

interleaving the plurality of encoded data segments into the plurality of encoded data slices.

18. The DS processing unit of claim 17, wherein the at least one processing module is operable to encode the plurality of data segments using the error encoding dispersal function to generate the plurality of encoded data segments by:

arranging data blocks of one data segment of the plurality of data segments into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix into an encoded data segment.