Meeting storage requirements with limited storage resources

• US 10,169,369 B2
• Filed: 11/13/2017
• Issued: 01/01/2019
• Est. Priority Date: 07/01/2013
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Johnson, Trent W., Baptist, Andrew D., Volvovski, Ilya, Quigley, John
• Primary Examiner(s): Tran, Anhtai V
• Application Number:
  US15/810,426
• Publication Number:
  US 20180067964A1
• Time in Patent Office: 414 Days
• Field of Search: None
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A managing unit included in a distributed storage network (DSN) determines resource availability information. The resource availability information includes information indicating a number of memory devices available to be used for storage of encoded data slices. The management unit then chooses dispersal parameters based on the resource availability information. The dispersal parameters are chosen to satisfy a performance threshold associated with storage of the encoded data slices in the DSN, but subject to a constraint requiring a pillar width associated with the encoded data slices to be no larger than the number of memory devices available. The management unit creates a number of storage units, wherein the number of storage units is constrained to be greater than the pillar width, and wherein one or more of the number of memory devices is assigned to each of the number of storage units. The management unit then instantiates a DSN memory that includes the number of storage units.

Claims

1. A method for use in a distributed storage network (DSN), the method comprising:

determining, at a managing unit, resource availability information, the resource availability information including information indicating a number of memory devices available to be used for storage of encoded data slices;

based on the resource availability information, choosing, at the management unit, dispersal parameters satisfying a performance threshold associated with storage of the encoded data slices in the DSN, subject to a constraint requiring a pillar width associated with the encoded data slices to be no larger than the number of memory devices available;

creating, at the management unit, a number of storage units based on the dispersal parameters, wherein the number of storage units is constrained to be greater than the pillar width, and wherein one or more of the number of memory devices is assigned to each of the number of storage units; and

instantiating, by the management unit, a DSN memory that includes the number of storage units.

2. The method of claim 1, further comprising:

assigning DSN address ranges to the number of storage units; and

allocating sub-DSN address ranges to particular memory devices of the number of storage units.

3. The method of claim 2, further comprising:

attempting to equalize an amount of total storage allocated to each of the number of storage units by assigning the DSN address ranges based on utilization of each of the number of storage units.

4. The method of claim 2, further comprising:

allocating the sub-DSN address ranges to the particular memory devices based on utilization of the particular memory devices.

5. The method of claim 1, wherein instantiating the DSN memory includes:

generating resource assignment information including the dispersal parameters; and

transmitting the resource assignment information to the number of storage units.

6. The method of claim 5, further comprising:

redistributing the resource assignment information throughout the DSN, including to devices other than the number of storage units.

7. The method of claim 1, further comprising:

choosing dispersal parameters to minimize shared hardware resources.

8. A computing device for use in a distributed storage network (DSN), the computing device comprising:

a processor;

memory operably associated with the processor and storing a program of instructions, upon execution by the processor, causing the processor to implement a managing unit configured to;

determine resource availability information, the resource availability information including information indicating a number of memory devices available to be used for storage of encoded data slices;

based on the resource availability information, choose dispersal parameters satisfying a performance threshold associated with storage of the encoded data slices in the DSN, subject to a constraint requiring a pillar width associated with the encoded data slices to be no larger than the number of memory devices available;

create a number of storage units based on the dispersal parameters, wherein the number of storage units is constrained to be greater than the pillar width, and wherein one or more of the number of memory devices is assigned to each of the number of storage units; and

instantiate a DSN memory that includes the number of storage units.

9. The computing device of claim 8, the managing unit further configured to:

assign DSN address ranges to the number of storage units; and

allocate sub-DSN address ranges to particular memory devices of the number of storage units.

10. The computing device of claim 9, the managing unit further configured to:

attempt to equalize an amount of total storage allocated to each of the number of storage units by assigning the DSN address ranges based on utilization of each of the number of storage units.

11. The computing device of claim 9, the managing unit further configured to:

allocate the sub-DSN address ranges to the particular memory devices based on utilization of the particular memory devices.

12. The computing device of claim 8, the managing unit further configured to instantiate the DSN memory by:

generating resource assignment information including the dispersal parameters; and

transmitting the resource assignment information to the number of storage units.

13. The computing device of claim 12, the managing unit further configured to:

redistribute the resource assignment information throughout the DSN, including to devices other than the number of storage units.

14. The computing device of claim 8, wherein:

the performance threshold includes at least one of mean time to data loss, or write availability.

15. A distributed storage network (DSN) comprising:

a plurality of memory devices configured to store error encoded data slices, wherein the error encoded data slices are associated with a pillar width; and

a managing unit including a processor and associated memory, the managing unit configured to;

determine resource availability information, the resource availability information including information indicating a number of memory devices available to be used for storage of the error encoded data slices;

based on the resource availability information, choose dispersal parameters satisfying a performance threshold associated with storage of the error encoded data slices in the DSN, the dispersal parameters being chosen subject to a constraint requiring the pillar width to be no larger than the number of memory devices available;

create a number of storage units based on the dispersal parameters, wherein the number of storage units is constrained to be greater than the pillar width, and wherein one or more of the number of memory devices is assigned to each of the number of storage units; and

instantiate a DSN memory that includes the number of storage units.

16. The distributed storage network (DSN) of claim 15, the managing unit further configured to:

assign DSN address ranges to the number of storage units; and

allocate sub-DSN address ranges to particular memory devices of the number of storage units.

17. The distributed storage network (DSN) of claim 16, the managing unit further configured to:

assign the DSN address ranges based on utilization of each of the number of storage units; and

allocate the sub-DSN address ranges to the particular memory devices based on utilization of the particular memory devices.

18. The distributed storage network (DSN) of claim 15, the managing unit further configured to instantiate the DSN memory by:

generating resource assignment information including the dispersal parameters; and

transmitting the resource assignment information to the number of storage units.

19. The distributed storage network (DSN) of claim 18, wherein:

the DSN includes a plurality of computing devices in addition to the managing unit; and

the managing unit further configured to redistribute the resource assignment information throughout the DSN, including to the plurality of computing devices.

20. The distributed storage network (DSN) of claim 15, wherein choosing the dispersal parameters includes:

choosing the pillar width so that an estimated performance satisfies both mean time to data loss and write availability requirements.