STORING DATA UTILIZING A MAXIMUM ACCESSIBILITY APPROACH IN A DISPERSED STORAGE NETWORK

• US 20180101308A1
• Filed: 12/13/2017
• Published: 04/12/2018
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin, Resch, Jason K.
• Granted Patent: US 10,331,384 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/306 User profiles

Abstract

A method for execution by a dispersed storage and task (DST) processing unit of a dispersed storage network includes determining to store data in a storage pool utilizing a maximum accessibility approach. A storage unit performance factor is determined for a plurality of storage units of the storage pool. A number of instances of data storage per storage unit is established based on the storage unit performance factor. A replication factor across the plurality of storage units of the storage pool is also established. A total number C of storage instances for the data is determined based on the number of instances of data storage per storage unit and the replication factor. C number of source names for C storage instances of the data are generated. Storage of the C storage instances of the data is facilitated.

Claims

1. A method for execution by a dispersed storage and task (DST) processing unit that includes a processor, the method comprises:

determining to store data in a storage pool utilizing a maximum accessibility approach;

determining a storage unit performance factor for a plurality of storage units of the storage pool;

establishing a number of instances of data storage per storage unit based on the storage unit performance factor;

establishing a replication factor across the plurality of storage units of the storage pool;

determining a total number C of storage instances for the data based on the number of instances of data storage per storage unit and the replication factor;

generating C number of source names for C storage instances of the data; and

facilitating storage of the C storage instances of the data.

2. The method of claim 1, wherein determining to store the data in a storage pool utilizing a maximum accessibility approach includes determining the data exceeds an access frequency threshold and further includes determining that available storage capacity is greater than a minimum storage capacity threshold level.

3. The method of claim 1, wherein determining the storage unit performance factor includes dividing a storage unit throughput level by a memory device throughput level.

4. The method of claim 1, wherein the number of instances of data storage per storage unit is established as a minimum of a number of memory devices per storage unit and the storage unit performance factor.

5. The method of claim 1, wherein establishing the replication factor includes calculating the replication factor by dividing a number of storage units of the storage pool by an information dispersal algorithm (IDA) width.

6. The method of claim 1, wherein determining the total number C of storage instances includes calculating the total number C of storage instances by multiplying the number of instances of data storage by the replication factor.

7. The method of claim 1, wherein generating C number of source names for the C storage instances of the data includes:

generating a first source name for the data;

identifying a DSN address range for the storage pool;

generating further source name space by spacing source names apart by the DSN address range divided by C across the DSN address range starting with the first source name; and

updating a directory to associate the first source name and the value of C.

8. The method of claim 7, further comprising:

determining to recover the data;

accessing the directory to recover the first source name and the value of C;

pseudo-randomly selecting one of the C storage instances to recover;

calculating a source name corresponding to the selected one of the C storage instances based on the first source name and the value of C; and

recovering the data from the selected one of the C storage instances.

9. The method of claim 1, wherein facilitating storage of the C storage instances of the data includes, for each storage instance of the C storage instances:

generating a plurality of sets of slice names based on a corresponding sourcing;

dispersed storage error encoding the data to produce a plurality of sets of encoded data slices corresponding to the plurality of sets of slice names;

generating a plurality of write slice requests that includes the plurality of sets of slice names and the corresponding plurality of sets of encoded data slices; and

sending the plurality of write slice requests to a set of storage units of the plurality of storage units, wherein the set of storage units is associated with the storage instance.

10. A processing system of a dispersed storage and task (DST) processing unit comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

determine to store data in a storage pool utilizing a maximum accessibility approach;

determine a storage unit performance factor for a plurality of storage units of the storage pool;

establish a number of instances of data storage per storage unit based on the storage unit performance factor;

establish a replication factor across the plurality of storage units of the storage pool;

determine a total number C of storage instances for the data based on the number of instances of data storage per storage unit and the replication factor;

generate C number of source names for C storage instances of the data; and

facilitate storage of the C storage instances of the data.

11. The processing system of claim 10, wherein determining to store the data in a storage pool utilizing a maximum accessibility approach includes determining the data exceeds an access frequency threshold and further includes determining that available storage capacity is greater than a minimum storage capacity threshold level.

12. The processing system of claim 10, wherein determining the storage unit performance factor includes dividing a storage unit throughput level by a memory device throughput level.

13. The processing system of claim 10, wherein the number of instances of data storage per storage unit is established as a minimum of a number of memory devices per storage unit and the storage unit performance factor.

14. The processing system of claim 10, wherein establishing the replication factor includes calculating the replication factor by dividing a number of storage units of the storage pool by an information dispersal algorithm (IDA) width.

15. The processing system of claim 10, wherein determining the total number C of storage instances includes calculating the total number C of storage instances by multiplying the number of instances of data storage by the replication factor.

16. The processing system of claim 10, wherein generating C number of source names for the C storage instances of the data includes:

generating a first source name for the data;

identifying a DSN address range for the storage pool;

generating further source name space by spacing source names apart by the DSN address range divided by C across the DSN address range starting with the first source name; and

updating a directory to associate the first source name and the value of C.

17. The processing system of claim 16, wherein the operational instructions, when executed by the at least one processor, further cause the processing system to:

determining to recover the data;

accessing the directory to recover the first source name and the value of C;

pseudo-randomly selecting one of the C storage instances to recover;

calculating a source name corresponding to the selected one of the C storage instances based on the first source name and the value of C; and

recovering the data from the selected one of the C storage instances.

18. The processing system of claim 10, wherein facilitating storage of the C storage instances of the data includes, for each storage instance of the C storage instances:

generating a plurality of sets of slice names based on a corresponding sourcing;

dispersed storage error encoding the data to produce a plurality of sets of encoded data slices corresponding to the plurality of sets of slice names;

generating a plurality of write slice requests that includes the plurality of sets of slice names and the corresponding plurality of sets of encoded data slices; and

sending the plurality of write slice requests to a set of storage units of the plurality of storage units, wherein the set of storage units is associated with the storage instance.

19. A computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

determine to store data in a storage pool utilizing a maximum accessibility approach;

determine a storage unit performance factor for a plurality of storage units of the storage pool;

establish a number of instances of data storage per storage unit based on the storage unit performance factor;

establish a replication factor across the plurality of storage units of the storage pool;

determine a total number C of storage instances for the data based on the number of instances of data storage per storage unit and the replication factor;

generate C number of source names for C storage instances of the data; and

facilitate storage of the C storage instances of the data.

20. The computer readable storage medium of claim 19, wherein establishing the replication factor includes calculating the replication factor by dividing a number of storage units of the storage pool by an information dispersal algorithm (IDA) width.