Allocating data based on memory device performance in a dispersed storage network

• US 10,540,230 B2
• Filed: 01/30/2019
• Issued: 01/21/2020
• Est. Priority Date: 03/29/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg R., Resch, Jason K.
• Primary Examiner(s): Pollack, Melvin H
• Application Number:
  US16/262,416
• Publication Number:
  US 20190163572A1
• Time in Patent Office: 356 Days
• Field of Search: 709224
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H03M 13/373 : with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
• CPC Class Codes: H03M 13/616 : Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a dispersed storage and task (DST) processing unit includes generating a request for mapping parameters for transmission a storage unit in a dispersed storage network (DSN) and receiving mapping parameter data in response. Namespace mapping data is generated, indicating a mapping of possible slice names to a plurality of memory devices of the storage unit by performing a first deterministic mapping function based on the mapping parameter data. Slice name subset data is generated, indicating a subset of the possible slice names based on a health status indicators corresponding to the plurality of memory devices. A slice name corresponding to a first encoded slice of a first data object is selected from the subset indicated by the slice name subset data. A first write request that includes the first encoded slice is generated for transmission via the network to the storage unit.

Claims

1. A method for execution by a computing device that includes a processor, the method comprises:

receiving first mapping parameter data from a first one of a plurality of storage units in a dispersed storage network (DSN);

generating first namespace mapping data that indicates a mapping of a plurality of possible slice names to a first plurality of memory devices of the first one of the plurality of storage units by performing a first deterministic mapping function based on the first mapping parameter data;

generating slice name subset data indicating a subset of the plurality of possible slice names based on a plurality of health status indicators corresponding to the first plurality of memory devices;

selecting a first slice name corresponding to a first encoded slice of a first data object from the subset indicated by the slice name subset data; and

generating a first write request for transmission via the network that includes the first encoded slice to the first one of the plurality of storage units.

2. The method of claim 1, further comprising receiving memory device status data from the one of the plurality of storage units via the network that includes the plurality of health status indicators.

3. The method of claim 2, wherein the memory device status data is received from the one of the plurality of storage units in response to a first request for mapping parameters.

4. The method of claim 1, wherein generating the slice name subset data includes comparing each of the plurality of health status indicators to a health status threshold, and wherein a one of the plurality of possible slice names is included in the subset when its corresponding health status indicator compares favorably to the health status threshold.

5. The method of claim 1, wherein the first deterministic mapping function is based on a Decentralized Agreement Protocol (DAP).

6. The method of claim 1, wherein the first deterministic mapping function corresponds to a mapping algorithm utilized by the one of the plurality of storage units, and wherein the first encoded slice is stored in a one of the first plurality of memory devices of the one of the plurality of storage units based on performing the mapping algorithm on the first encoded slice.

7. The method of claim 1, further comprising:

selecting a plurality of selected slice names that includes the first slice name, each corresponding to each of a plurality of encoded slices corresponding to the first data object based on the slice name subset data; and

generating a plurality of write requests that includes the first write request, each for transmission via the network to a corresponding one of the plurality of storage units, wherein each of the plurality of write requests indicates a corresponding one of the plurality of encoded slices;

wherein a first request for mapping parameters is transmitted to the plurality of storage units and each of a plurality of mapping parameter data, which includes the first mapping parameter data, is received from each of the plurality of storage units;

wherein the first namespace mapping data indicates, for each of the plurality of storage units, the corresponding mapping of a plurality of possible slice names to the first plurality of memory devices of the each of the plurality of storage units; and

wherein the slice name subset data indicates a plurality of subsets of the plurality of possible slice names for each of the plurality of storage units based on the plurality of health status indicators corresponding to the first plurality of memory devices corresponding to each of the plurality of storage units.

8. The method of claim 7, further comprising performing an encoding function on the first data object based on the slice name subset data to generate the plurality of encoded slices, wherein the corresponding plurality of selected slice names are also generated by the encoding function, and wherein the each of the corresponding plurality of selected slice names are included in one of the plurality of subsets indicated by the slice name subset data.

9. The method of claim 1, wherein the first deterministic mapping function is one of a plurality of deterministic mapping functions indicated by the first mapping parameter data.

10. The method of claim 1, further comprising:

receiving an access request from a requesting entity via the network indicating a second encoded slice stored in the DSN;

generating a second request for mapping parameters for transmission via a network to a second one of a plurality of storage units in the DSN and receiving second mapping parameter data in response;

generating slice location data that indicates a one of a second plurality of memory devices of the second one of the plurality of storage units corresponding to a location where the second encoded slice is stored by performing a second deterministic mapping function on a second slice name corresponding to the second encoded slice based on the second mapping parameter data; and

performing the access request when a health status indicator corresponding to the second one of the second plurality of memory devices compares favorably to a health status threshold.

11. The method of claim 10, further comprising:

generating a low-health notification for transmission to the requesting entity via the network indicating that the access request will not be performed and foregoing the access request when the health status indicator corresponding to the second one of the second plurality of memory devices compares unfavorably to the health status threshold.

12. The method of claim 1, wherein a first request for mapping parameters is generated in response to a slice reallocation requirement, and the method further comprises:

generating reallocation data in response to the slice name subset data indicating that an original slice name of the first encoded slice is not included in the subset, wherein the reallocation data indicates a new slice name to replace the original slice name of the first encoded slice;

wherein the first write request indicates the original slice name and a request to remove the first encoded slice from a memory device corresponding to old slice name.

13. The method of claim 12, wherein the slice reallocation requirement is received via the network.

14. The method of claim 12, wherein the slice reallocation requirement is based on a predetermined time interval elapsing.

15. The method of claim 12, wherein the slice reallocation requirement is based on performance data from past access to the plurality of storage units.

16. The method of claim 12, further comprising:

transmitting a request for memory device status data from the first one of the plurality of storage units via the network in response to the slice reallocation requirement; and

receiving plurality of health status indicators via the network in response to the request for memory device status data.

17. The method of claim 1, further comprising:

generating reallocation data in response to the size of the subset indicated by the slice name subset data comparing unfavorably to a required healthy memory device count, wherein the reallocation data indicates a plurality of encoded slices stored in the one of the plurality of storage units, each with a corresponding original slice name that is not included in the subset indicated by the slice name subset data, wherein the reallocation data further indicates a plurality of new slice names, each included in the subset indicated by the slice name subset data, and wherein each of the plurality of encoded slices is assigned a corresponding one of the plurality of new slice names; and

generating a second plurality of write requests, each for transmission via the network to the plurality of storage units, wherein each of the plurality of write requests indicates that a corresponding one of the plurality of encoded slices be moved from an old one of the first plurality of memory devices corresponding to the original slice name to a new one of the first plurality of memory devices corresponding to the new slice name.

18. The method of claim 17, wherein each of the plurality of new slice names are determined based on performing an encoding function on one of a plurality of data objects of the corresponding one of the plurality of encoded slices based on the slice name subset data.

19. A processing system of a computing device comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

receive first mapping parameter data from a first one of a plurality of storage units in a dispersed storage network (DSN);

generate first namespace mapping data that indicates a mapping of a plurality of possible slice names to a first plurality of memory devices of the first one of the plurality of storage units by performing a first deterministic mapping function based on the first mapping parameter data;

generate slice name subset data indicating a subset of the plurality of possible slice names based on a plurality of health status indicators corresponding to the first plurality of memory devices;

select a first slice name corresponding to a first encoded slice of a first data object from the subset indicated by the slice name subset data; and

generate a first write request for transmission via the network that includes the first encoded slice to the first one of the plurality of storage units.

20. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

receive first mapping parameter data from a first one of a plurality of storage units in a dispersed storage network (DSN);

generate first namespace mapping data that indicates a mapping of a plurality of possible slice names to a first plurality of memory devices of the first one of the plurality of storage units by performing a first deterministic mapping function based on the first mapping parameter data;

generate slice name subset data indicating a subset of the plurality of possible slice names based on a plurality of health status indicators corresponding to the first plurality of memory devices;

select a first slice name corresponding to a first encoded slice of a first data object from the subset indicated by the slice name subset data; and

generate a first write request for transmission via the network that includes the first encoded slice to the first one of the plurality of storage units.