Determining slices used in a reconstruction

• US 10,296,404 B2
• Filed: 11/04/2016
• Issued: 05/21/2019
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Abhijeet, Kumar, Baptist, Andrew D., Gladwin, S. Christopher, Resch, Jason K.
• Primary Examiner(s): Schell, Joseph O
• Application Number:
  US15/343,820
• Publication Number:
  US 20170052848A1
• Time in Patent Office: 928 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/073 in a memory management context, e.g. virtual memory or cache management memory management G06F12/00; testing of static memory units G11C29/00
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1612 where the redundant component is persistent storage
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 : Management of blocks
• CPC Class Codes: G06F 3/0644 : Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for a dispersed storage network (DSN) begins by receiving a read data object request from for a data object stored as a plurality of sets of encoded data slices in a set of storage units. The method continues by selecting a read threshold number of encoded data slices for retrieval. The method continues by issuing read slice requests to at least some of the storage units of the storage set to recover the selected read threshold number of encoded data slices. The method continues by receiving read slice responses from at least some of the storage units and selecting a decode threshold number of encoded data slices. The method continues by decoding the selected decode threshold number of encoded data slices to reproduce the data and generating audit information based on the selected slices for retrieval, the received slice, and the selected slices for decoding.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a read data object request from for a data object stored as a plurality of sets of encoded data slices in a set of storage units;

selecting, for each of the plurality of sets of encoded data slices, a read threshold number of encoded data slices for retrieval;

issuing read slice requests to at least some storage units of the set of storage units to recover the selected read threshold number of encoded data slices;

receiving read slice responses from at least some of the storage units, the read slice responses including received encoded data slices of the selected read threshold number of encoded data slices;

selecting, for each of the received read slice responses of a set of the plurality of sets of encoded data slices, a decode threshold number of encoded data slices;

decoding the selected decode threshold number of encoded data slices to reproduce the data object; and

generating audit information based on the selected read threshold number of encoded data slices for retrieval, the received read slice responses, and the selected decode threshold number of encoded data slices; and

sending the reproduced data object and audit information to a requesting entity, wherein the audit information includes;

an identifier of the data object, identifiers of the selected read threshold number of encoded data slices, identifiers of received encoded data slices, and identifiers of the selected decode threshold number of encoded data slices.

2. The method of claim 1 further comprises facilitating storage of the audit information in one or more of a local memory or the set of storage units.

3. The method of claim 1, wherein the selecting, for each of the plurality of sets of encoded data slices, a read threshold number of encoded data slices for retrieval is based on any of:

a performance level, an availability level, a predetermination a temporarily stored identifier, or slice names of the selected read threshold number of encoded data slices.

4. The method of claim 1, wherein the issuing read slice requests includes:

identifying storage units associated with the read threshold number of encoded data slices, generating the read slice requests, and sending the read slice requests to the identified storage units.

5. The method of claim 1 further comprises temporarily storing identifiers of received encoded data slices.

6. The method of claim 1, wherein the selecting, for each of the receiving slice responses of a set of the plurality of sets of encoded data slices is based on one or more of:

a decode threshold number of encoded data slices first received, random, or a list.

7. The method of claim 1 further comprises temporarily storing identifiers of the selected decode threshold number of encoded data slices.

8. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

receive a read data object request from for a data object stored as a plurality of sets of encoded data slices in a set of storage units;

select, for each of the plurality of sets of encoded data slices, a read threshold number of encoded data slices for retrieval;

issue read slice requests to at least some storage units of the set of storage units to recover the selected read threshold number of encoded data slices;

receive read slice responses from at least some of the storage units, the read slice responses including received encoded data slices of the selected read threshold number of encoded data slices;

select, for each of the received read slice responses of a set of the plurality of sets of encoded data slices, a decode threshold number of encoded data slices;

decode the selected decode threshold number of encoded data slices to reproduce the data object; and

generate audit information based on the selected read threshold number of encoded data slices for retrieval, the received read slice responses, and the selected decode threshold number of encoded data slices; and

send the reproduced data object and audit information to a requesting entity, wherein the audit information includes;

an identifier of the data object, identifiers of the selected read threshold number of encoded data slices, identifiers of received encoded data slices, and identifiers of the selected decode threshold number of encoded data slices.

9. The computing device of claim 8 further comprises facilitating storage of the audit information in one or more of a local memory or the set of storage units.

10. The computing device of claim 8, wherein the selecting, for each of the plurality of sets of encoded data slices, a read threshold number of encoded data slices for retrieval is based on any of:

a performance level, an availability level, a predetermination a temporarily stored identifier, or slice names of the selected read threshold number of encoded data slices.

11. The computing device of claim 8, wherein the issuing read slice requests includes:

identifying storage units associated with the read threshold number of encoded data slices, generating the read slice requests, and sending the read slice requests to the identified storage units.

12. The computing device of claim 8 further comprises temporarily storing identifiers of received encoded data slices.

13. The computing device of claim 8 further comprises temporarily storing identifiers of the selected decode threshold number of encoded data slices.

14. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a read data object request from a user device to reproduce a data object, wherein the data object is dispersed storage error encoded to produce a plurality of sets of encoded data slices that are stored in a DSN storage set and wherein at least a decode threshold number of encoded data slices are required from each set to recover the data object;

selecting for each of the plurality of sets of encoded data slices, a read threshold number of encoded data slices for retrieval;

issuing read slice requests to at least some storage units of the DSN storage set to recover the selected read threshold number of encoded data slices;

receiving read slice responses from at least some storage units in the DSN storage set;

selecting, for each of the read slice responses of a set of the plurality of sets of encoded data slices, a decode threshold number of encoded data slices;

for each set of encoded data slices, decoding the selected decode threshold number of encoded data slices to reproduce the data object; and

sending the reproduced data object and audit information to the user device, wherein the audit information includes, for each set of encoded data slices, identifiers of the selected read threshold number of encoded data slices, identifiers of received encoded data slices, and identifiers of the selected decode threshold number of encoded data slices.