Determining respective mappings for logically defined dispersed storage units

• US 10,318,189 B2
• Filed: 11/10/2016
• Issued: 06/11/2019
• Est. Priority Date: 12/30/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Iljazi, Ilir
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US15/347,873
• Publication Number:
  US 20170192699A1
• Time in Patent Office: 943 Days
• Field of Search: 709217, 714763, 711162
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 12/0684 with feedback, e.g. presence or absence of unit detected by addressing, overflow detection
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0888 using selective caching, e.g. bypass
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0616 in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0652 : Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0653 : Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 : Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0661 : Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 : at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 : using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 43/0852 : Delays
• CPC Class Codes: H04L 43/0876 : Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 43/0888 : Throughput
• CPC Class Codes: H04L 43/16 : Threshold monitoring
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods for assigning data storage resources are disclosed. The methods includes obtaining memory resource availability information for a plurality of physical memory devices of a dispersed storage network, determining a number of logical storage units for the dispersed storage network, determining a memory capacity level for each of the logical storage units, determining a mapping in accordance with a mapping approach of at least some of the plurality of physical memory devices to achieve a corresponding memory capacity level, and when accessing encoded data slices associated with the logical storage unit by an accessing entity, communicating slice access messages with a dispersed storage unit associated with the logical storage unit. A dispersed storage managing unit and dispersed storage network are also disclosed.

Claims

1. A method of accessing data in a dispersed storage network, the dispersed storage network including a plurality of dispersed storage units, the plurality of dispersed storage units each including a respective plurality of physical memory devices, the method comprising:

obtaining memory resource availability information for the respective plurality of physical memory devices of the plurality of dispersed storage units;

determining a number of logical storage units for the dispersed storage network to produce a determined number of logical storage units, wherein the number of logical storage units include an association of a plurality of physical memory devices of two or more physical storage units with at least one of the physical storage units providing a hosting function for the logical storage units, and wherein the number of logical storage units includes a minimum of an information dispersal algorithm (IDA) width number;

determining respective memory capacity levels for the determined number of logical storage units; and

determining respective mappings for the determined number of logical storage units using the memory resource availability for the respective plurality of physical memory devices to achieve the respective memory capacity levels, wherein the respective mappings each involve a respective group of at least two physical memory devices of the plurality of dispersed storage units.

2. The method of claim 1, wherein the step of obtaining memory resource availability information for the respective plurality of physical memory devices of the plurality of dispersed storage units further comprises interpreting resource availability information received from the plurality of dispersed storage units.

3. The method of claim 2, wherein the memory resource availability information includes one or more of a memory device type, a memory device capacity level, a memory device access delay performance level, and a security factor level.

4. The method of claim 1, wherein the step of determining respective mappings for the determined number of logical storage units using the memory resource availability for the respective plurality of physical memory devices to achieve the respective memory capacity levels further comprises determining respective mappings for the determined number of logical storage units in accordance with a mapping approach of at least some of the respective plurality of physical memory devices of the plurality of dispersed storage units.

5. The method of claim 4, wherein the mapping approach includes one or more of avoiding using more than one of the respective physical memory devices of any of the plurality of dispersed storage units, utilizing the respective plurality of physical memory devices of one of the plurality of dispersed storage units selected as a host physical storage unit first, and utilizing the respective plurality of physical memory devices of the plurality of dispersed storage units with fast network connectivity first.

6. The method of claim 1, further comprising accessing encoded data slices associated with a first logical storage unit of the determined number of logical storage units by communicating with a first dispersed storage unit of the plurality of dispersed storage units associated with the first logical storage unit.

7. The method of claim 1, wherein the step of determining the number of logical storage units for the dispersed storage network to produce the determined logical storage units further comprises one or more of interpreting system registry information, interpreting a request, and determining the number of logical storage units within a range from the minimum of an informational dispersal algorithm width number to a maximum of the total number of physical memory devices of the plurality of dispersed storage units that are available.

8. The method of claim 1, wherein the step of determining respective memory capacity levels for the determined number of logical storage units further comprises one or more of interpreting system registry information, interpreting a request, and multiplying the determined number of logical storage units by an average memory capacity level of the physical memory devices of the plurality of dispersed storage units.

9. The method of claim 1, wherein the step of determining respective mappings for the determined number of logical storage units using the memory resource availability for the respective plurality of physical memory devices to achieve the respective memory capacity levels further comprises one or more of selecting a first physical storage unit of a first respective group of least two physical memory devices to host a first logical storage unit of the determined number of logical storage units, selecting a second respective group of at least two physical memory devices to serve as logical memories of a second logical storage unit of the determined number of logical storage units, and mapping a unique portion of an assigned dispersed storage network sub-address range of a third logical storage unit of the determined number of logical storage units to a third respective group of at least two physical memory devices.

10. A dispersed storage managing unit for use in a dispersed storage network, the dispersed storage network including a plurality of dispersed storage units, the plurality of dispersed storage units each including a respective plurality of physical memory devices, the dispersed storage managing unit comprising:

a communications interface;

a memory; and

a computer processor;

where the memory includes instructions for causing the computer processor to;

obtain memory resource availability information for the respective plurality of physical memory devices of the plurality of dispersed storage units;

determine a number of logical storage units for the dispersed storage network to produce a determined number of logical storage units, wherein the number of logical storage units includes an association of a plurality of physical memory devices of two or more physical storage units with at least one of the physical storage units providing a hosting function for the logical storage units, and wherein the number of logical storage units includes a minimum of an information dispersal algorithm (IDA) width number;

determine respective memory capacity levels for the determined number of logical storage units; and

determine respective mappings for the determined number of logical storage units using the memory resource availability for the respective plurality of physical memory devices to achieve the respective memory capacity levels, wherein the respective mappings each involve a respective group of at least two physical memory devices of the plurality of dispersed storage units.

11. The dispersed storage managing unit of claim 10, wherein the memory further comprises instructions for causing the computer processor to interpret resource availability information received from the plurality of dispersed storage units.

12. The dispersed storage managing unit of claim 11, wherein the memory resource availability information includes one or more of a memory device type, a memory device capacity level, a memory device access delay performance level, and a security factor level.

13. The dispersed storage managing unit of claim 10, wherein the memory further comprises instructions for causing the computer processor to determine respective mappings for the determined number of logical storage units in accordance with a mapping approach of at least some of the respective plurality of physical memory devices of the plurality of dispersed storage units.

14. The dispersed storage managing unit of claim 13, wherein the mapping approach one or more of avoids using more than one of the respective physical memory devices of any of the plurality of dispersed storage units, utilizes the respective plurality of physical memory devices of one of the plurality of dispersed storage units selected as a host physical storage unit first, and utilizes the respective plurality of physical memory devices of the plurality of dispersed storage units with fast network connectivity first.

15. The dispersed storage managing unit of claim 10, wherein the memory further comprises instructions for causing the computer processor to one or more of interpret system registry information, interpret a request, and determine the number of logical storage units within a range from a minimum of the informational dispersal algorithm width number to a maximum of the total number of physical memory devices of the plurality of dispersed storage units that are available.

16. The dispersed storage managing unit of claim 10, wherein the memory further comprises instructions for causing the computer processor to one or more of interpret system registry information, interpret a request, and multiply the determined number of logical storage units by an average memory capacity level of the physical memory devices of the plurality of dispersed storage units.

17. The dispersed storage managing unit of claim 10, wherein the memory further comprises instructions for causing the computer processor to one or more of select a first physical storage unit of a first respective group of least two physical memory devices to host a first logical storage unit of the determined number of logical storage units, select a second respective group of at least two physical memory devices to serve as logical memories of a second logical storage unit of the determined number of logical storage units, and map a unique portion of an assigned dispersed storage network sub-address range of a third logical storage unit of the determined number of logical storage units to a third respective group of at least two physical memory devices.

18. A dispersed storage network comprising:

a plurality of dispersed storage units, the plurality of dispersed storage units each including a respective plurality of physical memory devices; and

a dispersed storage managing unit, the dispersed storage managing unit including;

a communications interface;

a memory; and

a computer processor;

where the memory includes instructions for causing the computer processor to;

obtain memory resource availability information for the respective plurality of physical memory devices of the plurality of dispersed storage units;

determine a number of logical storage units for the dispersed storage network to produce a determined number of logical storage units, wherein the number of logical storage units includes an association of a plurality of physical memory devices of two or more physical storage units with at least one of the physical storage units providing a hosting function for the logical storage units, and wherein the number of logical storage units includes a minimum of an information dispersal algorithm (IDA) width number;

determine respective memory capacity levels for the determined number of logical storage units; and

determine respective mappings for the determined number of logical storage units using the memory resource availability for the respective plurality of physical memory devices to achieve the respective memory capacity levels, wherein the respective mappings each involve a respective group of at least two physical memory devices of the plurality of dispersed storage units.

19. The dispersed storage network of claim 18, wherein the memory further comprises instructions for causing the computer processor to interpret resource availability information received from the plurality of dispersed storage units.

20. The dispersed storage network of claim 19, wherein the memory resource availability information includes one or more of a memory device type, a memory device capacity level, a memory device access delay performance level, and a security factor level.