Consolidating encoded data slices in read memory devices in a distributed storage network

• US 10,114,588 B2
• Filed: 01/05/2017
• Issued: 10/30/2018
• Est. Priority Date: 03/29/2016
• Status: Active Grant

• Current Assignee: Green Market Square Limited
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin
• Primary Examiner(s): Kim, Hong C
• Application Number:
  US15/399,022
• Publication Number:
  US 20170285999A1
• Time in Patent Office: 663 Days
• Field of Search: 711112, 711114, 711154, 707827, 709217
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 2221/2133 Verifying human interaction, e.g., Captcha
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: G06F 9/485 Task life-cycle, e.g. stopping, restarting, resuming execution G06F9/4881 takes precedence
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/373 : with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
• CPC Class Codes: H03M 13/616 : Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 61/10 : of different types
• CPC Class Codes: H04L 61/50 : Address allocation
• CPC Class Codes: H04L 63/08 : for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/104 : Grouping of entities
• CPC Class Codes: H04L 63/108 : when the policy decisions are valid for a limited amount of time
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/56 : Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A method begins by a processing module of a dispersed storage network (DSN) receiving a write request from a user to store encoded data slices and identifying a plurality of memory devices associated with the user. The method continues with the processing module selecting one or more memory devices based on whether the memory devices are read memory devices, write memory devices or read/write memory devices. The processing module then stores the encoded data slices in the selected memory devices and upon receiving a read request for the encoded data slices determines whether the encoded data slices have been consolidated in one or more read memories and when the encoded data slices have not been consolidated in one or more read memories, recovers the encoded data slices from one or more write memories of the plurality of memory devices.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a write request from a user to store encoded data slices;

identifying a plurality of memory devices associated with the user;

selecting one or more memory devices of the plurality of memory devices, based on a status of each of the memory devices of the plurality of memory devices, wherein the status of the one or more memory devices of the plurality of memory devices is based on the one or more memory devices of the plurality of memory devices being at least one of read memory devices, write memory devices or read/write memory devices;

storing the encoded data slices in the one or more selected memory devices of the plurality of memory devices;

receiving a read request for the encoded data slices;

determining whether the encoded data slices have been consolidated in one or more read memories; and

when the encoded data slices have not been consolidated in one or more read memories, recovering at least one of the encoded data slices from one or more write memories of the plurality of memory devices.

2. The method of claim 1, wherein the identifying a plurality of memory devices associated with the user is based on at least one of extracting an encoded data slice name from the write request, performing a deterministic function on the encoded data slice name to produce a list of memory identifiers, or performing a lookup based on the encoded data slice name to produce identities for each of the plurality of memories.

3. The method of claim 1, wherein the selecting further includes determining a corresponding status of each of the memory devices of the plurality of memory devices and identifying a memory associated with a status indicating availability for writing.

4. The method of claim 1, wherein the determining whether the encoded data slices have been consolidated in one or more read memories is based on at least one encoded data slice stored in a write only memory that has not been transferred to the read memory.

5. The method of claim 1, wherein the storing the encoded data slices in the one or more selected memory devices of the plurality of memory devices includes storing one or more encoded data slices in write only memory.

6. The method of claim 1 further comprises:

transferring, by the one or more processing modules, one or more encoded data slices from the write memory to the read memory and deleting the encoded data slices from the write memory subsequent to confirming that the transferring has successfully completed.

7. The method of claim 1, wherein the memory devices of the plurality of memory devices include at least read memory devices and write memory devices, and all the memory devices are addressable by encoded data slice names within a same encoded data slice name range.

8. The method of claim 7, wherein the same encoded data slice name range is used for at least two different storage units.

9. The method of claim 7, wherein the same encoded data slice name range is used for at least two different memory devices statuses.

10. The method of claim 7, wherein the read memory devices service all read requests for the encoded data slices and the write memory devices service all write requests for the encoded data slices.

11. A non-transitory computer readable medium comprises:

a first memory element that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to;

receive a write request from a user to store encoded data slices;

identify a plurality of memory devices associated with the user;

select one or more memory devices of the plurality of memory devices, based on a status of each of the memory devices of the plurality of memory devices, wherein the status of the one or more memory devices of the plurality of memory devices is based on the one or more memory devices of the plurality of memory devices being at least one of read memory devices, write memory devices or read/write memory devices;

store the encoded data slices in the one or more selected memory devices of the plurality of memory devices;

receive a read request for the encoded data slices;

determine whether the encoded data slices have been consolidated in one or more read memories; and

when the encoded data slices have not been consolidated in one or more read memories, recover at least one of the encoded data slices from one or more write memories of the plurality of memory devices.

12. The non-transitory computer readable medium of claim 11, further comprises:

identify the plurality of memory devices associated with the user based on at least one of extracting an encoded data slice name from the write request, performing a deterministic function on the encoded data slice name to produce a list of memory identifiers, or performing a lookup based on the encoded data slice name to produce identities for each of the plurality of memories.

13. The non-transitory computer readable medium of claim 11, wherein the select one or more memory devices of the plurality of memory devices further comprises:

determine a corresponding status of each of the memory devices of the plurality of memory devices and identify a memory associated with a status indicating availability for writing.

14. The non-transitory computer readable medium of claim 11, further comprises:

determine whether the encoded data slices have been consolidated in one or more read memories based on at least one encoded data slice stored in a write only memory that has not been transferred to the read memory.

15. The non-transitory computer readable medium of claim 11, wherein the store the encoded data slices in the one or more selected memory devices of the plurality of memory devices further comprises:

store one or more encoded data slices in write only memory.

16. The non-transitory computer readable medium of claim 11, further comprises:

transfer one or more encoded data slices from the write memory to the read memory; and

delete the encoded data slices from the write memory subsequent to confirming that the transferring has successfully completed.

17. The non-transitory computer readable medium of claim 11, wherein the memory devices of the plurality of memory devices include at least read memory devices and write memory devices, and all the memory devices are addressable by encoded data slice names within a same encoded data slice name range.

18. The non-transitory computer readable medium of claim 11, wherein the same encoded data slice name range is used for at least two different storage units.

19. The non-transitory computer readable medium of claim 11, wherein the same encoded data slice name range is used for at least two different memory devices statuses.

20. The non-transitory computer readable medium of claim 11, wherein the read memory devices service all read requests for the encoded data slices and the write memory devices service all write requests for the encoded data slices.