DISTRIBUTED STORAGE NETWORK FOR STORING A DATA OBJECT BASED ON STORAGE REQUIREMENTS

• US 20170285946A1
• Filed: 06/20/2017
• Published: 10/05/2017
• Est. Priority Date: 10/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Leggette, Wesley, Gladwin, S. Christopher, Dhuse, Greg, Baptist, Andrew
• Granted Patent: US 10,275,161 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/20 Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
• CPC Class Codes: H04N 21/218 Source of audio or video content , e.g. local disk arrays details of retrieval in video databases G06F16/739

Abstract

A distributed storage network (DSN) includes a user device and a plurality of DSN memories, wherein each of the DSN memories includes a plurality of storage units. The user device includes at least one network interface to the plurality of DSN memories and at least one processing module that is operable to determine one of the plurality of DSN memories for storing a data object based on a comparison of one or more storage requirements of the data object and one or more DSN attributes of the plurality of DSN memories.

Claims

1. A user device in a distributed storage network (DSN), comprises:

at least one network interface to the distributed storage network; and

at least one processing module that includes circuitry configured to;

determine to store a data object comprising a plurality of data segments;

determine one or more storage requirements for storing the data object in one of a plurality of DSN memories in the distributed storage network;

determine one or more DSN attributes associated with each of the plurality of DSN memories;

select one of the plurality of DSN memories to produce a selected DSN memory in response to a weighted comparison of the one or more storage requirements of the data object with the one or more DSN attributes for the selected DSN memory;

select one of a plurality of DSN processing units to produce a selected DSN processing unit;

transmit, via the at least one network interface, a store request message to the selected DSN processing unit to generate a plurality of encoded data slices using an error encoding dispersal function by encoding the plurality of data segments using the error encoding dispersal function to generate a plurality of encoded data segments and interleaving each of the plurality of encoded data segments into the plurality of encoded data slices; and

transmit, via the at least one network interface, the plurality of encoded data slices to the selected DSN memory.

2. The user device of claim 1, wherein the processing module is further operable to:

determine a priority level of the data object; and

access resource that includes storage requirements for the priority level of the data object.

3. The user device of claim 2, wherein a first storage requirement of the one or more storage requirements has a first higher priority level and a second storage requirement of the one or more storage requirements has a second lower priority level.

4. The user device of claim 3, wherein the processing module is further operable to:

allocate a greater weight to a first favorable comparison between a first DSN attribute and the first storage requirement with the first higher priority level; and

allocate a lesser weight to a second favorable comparison between a second DSN attribute and the second storage requirement with the second lower priority level.

5. The user device of claim 1, wherein the one or more DSN attributes includes one or more of the following:

cost, geographic location, geographic diversity, performance, reliability, latency, capacity, load and availability.

6. The user device of claim 1, wherein encoding at least one of the plurality of data segments using the error encoding dispersal function to generate one of the plurality of encoded data segments includes:

arranging data blocks of the at least one of the plurality of data segments into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix to generate the one of the plurality of encoded data segments.

7. The user device of claim 1, wherein the processing module is further operable to:

transmit, via the at least one network interface, an address update message to a DS managing unit to update a user vault with the selected DSN memory for the data object.

8. A method for use in a user device of a distributed storage network (DSN), the method comprises:

determining, via a system that includes processing circuitry, to store a data object comprising a plurality of data segments;

determining, via the system, one or more storage requirements for storing the data object in one of a plurality of DSN memories in the distributed storage network;

determining, via the system, one or more DSN attributes associated with each of the plurality of DSN memories;

selecting, via the system, one of the plurality of DSN memories to produce a selected DSN memory in response to a weighted comparison of the one or more storage requirements of the data object with the one or more DSN attributes for the selected DSN memory;

selecting, via the system, one of a plurality of DSN processing units to produce a selected DSN processing unit;

transmitting, via at least one network interface, a store request message to the selected DSN processing unit to generate a plurality of encoded data slices using an error encoding dispersal function by encoding the plurality of data segments using the error encoding dispersal function to generate a plurality of encoded data segments and interleaving each of the plurality of encoded data segments into the plurality of encoded data slices; and

transmitting, via the at least one network interface, the plurality of encoded data slices to the selected DSN memory.

9. The method of claim 8, further comprising:

determining a priority level of the data object; and

accessing a resource that includes storage requirements for the priority level of the data object.

10. The method of claim 9, wherein a first storage requirement of the one or more storage requirements has a first higher priority level and a second storage requirement of the one or more storage requirements has a second lower priority level.

11. The method of claim 10, further comprising:

allocating, via the system, a greater weight to a first favorable comparison between a first DSN attribute and the first storage requirement with the first higher priority level; and

allocating, via the system, a lesser weight to a second favorable comparison between a second DSN attribute and the second storage requirement with the second lower priority level.

12. The method of claim 11, wherein the one or more DSN attributes includes one or more of the following:

cost, geographic location, geographic diversity, performance, reliability, latency, capacity, load and availability.

13. The method of claim 8, wherein encoding at least one of the plurality of data segments using the error encoding dispersal function to generate one of the plurality of encoded data segments includes:

arranging data blocks of the at least one of the plurality of data segments into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix to generate the one of the plurality of encoded data segments.

14. The method of claim 8, further comprising:

transmitting, via the at least one network interface, an address update message to a DS managing unit to update a user vault with the selected DSN memory for the data object.

15. A non-transitory storage medium for use in a user device of a distributed storage network (DSN), that stores instructions, when executed by processing circuitry, that configure the processing circuitry to:

determine to store a data object comprising a plurality of data segments;

determine one or more storage requirements for storing the data object in one of a plurality of DSN memories in the distributed storage network;

determine one or more DSN attributes associated with each of the plurality of DSN memories;

select one of the plurality of DSN memories to produce a selected DSN memory in response to a weighted comparison of the one or more storage requirements of the data object with the one or more DSN attributes for the selected DSN memory;

select one of a plurality of DSN processing units to produce a selected DSN processing unit;

transmit, via at least one network interface, a store request message to the selected DSN processing unit to generate a plurality of encoded data slices using an error encoding dispersal function by encoding the plurality of data segments using the error encoding dispersal function to generate a plurality of encoded data segments and interleaving each of the plurality of encoded data segments into the plurality of encoded data slices; and

transmit, via the at least one network interface, the plurality of encoded data slices to the selected DSN memory.

16. The non-transitory storage medium of claim 15, wherein the instructions, when executed by processing circuitry, further configure the processing circuitry to:

determine a priority level of the data object; and

access a resource that includes storage requirements for the priority level of the data object.

17. The non-transitory storage medium of claim 16, wherein a first storage requirement of the one or more storage requirements has a first higher priority level and a second storage requirement of the one or more storage requirements has a second lower priority level.

18. The non-transitory storage medium of claim 17, wherein the instructions, when executed by processing circuitry, further configure the processing circuitry to:

allocate a greater weight to a first favorable comparison between a first DSN attribute and the first storage requirement with the first higher priority level; and

allocate a lesser weight to a second favorable comparison between a second DSN attribute and the second storage requirement with the second lower priority level.

19. The non-transitory storage medium of claim 18, wherein the one or more DSN attributes includes one or more of the following:

cost, geographic location, geographic diversity, performance, reliability, latency, capacity, load and availability.

20. The non-transitory storage medium of claim 15, wherein encoding at least one of the plurality of data segments using the error encoding dispersal function to generate one of the plurality of encoded data segments includes:

arranging data blocks of the at least one of the plurality of data segments into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix to generate the one of the plurality of encoded data segments.