Accessing serially stored data in a dispersed storage network

• US 10,409,772 B2
• Filed: 11/27/2017
• Issued: 09/10/2019
• Est. Priority Date: 02/27/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Wozniak, Ethan S., Baptist, Andrew D., Dhuse, Greg R., Volvovski, Ilya, Resch, Jason K.
• Primary Examiner(s): Chung, Phung M
• Application Number:
  US15/822,302
• Publication Number:
  US 20180075047A1
• Time in Patent Office: 652 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by a computing device includes determining to serially store data in a set of storage units utilizing a dispersed storage error coding function. An information dispersal algorithm (IDA) threshold number of received data blocks are encoded for every IDA threshold number of received data blocks of the data to produce sets of encoded data slices. Storage of the sets of encoded data slices in the set of storage units is facilitated, where each storage unit appends each of a plurality of received encoded data slices to a previously appended encoded data slice. A read offset is generated for the corresponding set of encoded data slices based on an identified portion of the data for recovery. A decode threshold number of data blocks corresponding to the read offset are retrieved and dispersed storage error decoded.

Claims

1. A method for execution by a computing device that includes a processor, the method comprises:

determining to serially store a data object in a set of storage units based on determining that a data size of the data object is greater than a serial storage threshold level;

encoding an information dispersal algorithm (IDA) threshold number of received data blocks for every IDA threshold number of received data blocks of the data object by performing a dispersed storage error coding function to produce a set of encoded data slices for the every IDA threshold number of received data blocks;

facilitating storage of the corresponding set of encoded data slices in the set of storage units for the every IDA threshold number of received data blocks, wherein each storage unit appends each of a plurality of received encoded data slices to a previously appended encoded data slice for storage;

identifying a portion of the data object for recovery from the set of storage units based on receiving a retrieval request;

generating a read offset for the corresponding set of encoded data slices based on the portion of the data object in response to identifying the portion of the data object;

retrieving a decode threshold number of data blocks of a set of data blocks corresponding to the read offset; and

dispersed storage error decoding the decode threshold number of data blocks to reproduce the portion of the data object.

2. The method of claim 1, wherein the computing device determines to serially store the data object in response to receiving the data object in a store data request.

3. The method of claim 1, wherein facilitating storage of a first set of encoded data slices corresponding to a first IDA threshold number of received data blocks includes issuing write slice requests to the set of storage units for the first set of encoded data slices, and wherein facilitating storage of at least one subsequent set of encoded data slices corresponding to at least one subsequent IDA threshold number of received data blocks includes issuing slice append requests for the at least one subsequent set of encoded data slices.

4. The method of claim 1, wherein the previously appended encoded data slice corresponds to an encoded data slice of the set of encoded data slices corresponding to a most recent IDA threshold number of received data blocks.

5. The method of claim 1, wherein identifying the portion of the data object includes interpreting the retrieval request to identify at least one IDA threshold number of received data blocks for recovery.

6. The method of claim 5, wherein the read offset is generated based on an offset number of IDA threshold number of received data blocks from a first IDA threshold number of appended encoded data slices to the identified at least one IDA threshold number of received data blocks for recovery.

7. The method of claim 1, wherein retrieving the decode threshold number of data blocks includes issuing read slice requests that include the read offset to the set of storage units and receiving at least one of the decode threshold number of data blocks in response.

8. A processing system of a computing device comprises:

at least one processor;

a memory that stores operational instructions, that when executed by the at least one processor cause the processing system to;

determine to serially store a data object in a set of storage units based on determining that a data size of the data object is greater than a serial storage threshold level;

encode an information dispersal algorithm (IDA) threshold number of received data blocks for every IDA threshold number of received data blocks of the data object by performing a dispersed storage error coding function to produce a set of encoded data slices for the every IDA threshold number of received data blocks;

facilitate storage of the corresponding set of encoded data slices in the set of storage units for the every IDA threshold number of received data blocks, wherein each storage unit appends each of a plurality of received encoded data slices to a previously appended encoded data slice for storage;

identify a portion of the data object for recovery from the set of storage units based on receiving a retrieval request;

generate a read offset for the corresponding set of encoded data slices based on the portion of the data object in response to identifying the portion of the data object;

retrieve a decode threshold number of data blocks of a set of data blocks corresponding to the read offset; and

dispersed storage error decoding the decode threshold number of data blocks to reproduce the portion of the data object.

9. The processing system of claim 8, wherein the processing system determines to serially store the data object in response to receiving the data object in a store data request.

10. The processing system of claim 8, wherein facilitating storage of a first set of encoded data slices corresponding to a first IDA threshold number of received data blocks includes issuing write slice requests to the set of storage units for the first set of encoded data slices, and wherein facilitating storage of at least one subsequent set of encoded data slices corresponding to at least one subsequent IDA threshold number of received data blocks includes issuing slice append requests for the at least one subsequent set of encoded data slices.

11. The processing system of claim 8, wherein the previously appended encoded data slice corresponds to an encoded data slice of the set of encoded data slices corresponding to a most recent IDA threshold number of received data blocks.

12. The processing system of claim 8, wherein identifying the portion of the data object includes interpreting the retrieval request to identify at least one IDA threshold number of received data blocks for recovery.

13. The processing system of claim 12, wherein the read offset is generated based on an offset number of IDA threshold number of received data blocks from a first IDA threshold number of appended encoded data slices to the identified at least one IDA threshold number of received data blocks for recovery.

14. The processing system of claim 8, wherein retrieving the decode threshold number of data blocks includes issuing read slice requests that include the read offset to the set of storage units and receiving at least one of the decode threshold number of data blocks in response.

15. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by a processing system of a dispersed storage network (DSN) that includes a processor and a memory, causes the processing system to;

determine to serially store a data object in a set of storage units based on determining that a data size of the data object is greater than a serial storage threshold level;

encode an information dispersal algorithm (IDA) threshold number of received data blocks for every IDA threshold number of received data blocks of the data object by performing a dispersed storage error coding function to produce a set of encoded data slices for the every IDA threshold number of received data blocks;

facilitate storage of the corresponding set of encoded data slices in the set of storage units for the every IDA threshold number of received data blocks, wherein each storage unit appends each of a plurality of received encoded data slices to a previously appended encoded data slice for storage;

identify a portion of the data object for recovery from the set of storage units based on receiving a retrieval request;

generate a read offset for the corresponding set of encoded data slices based on the portion of the data object in response to identifying the portion of the data object;

retrieve a decode threshold number of data blocks of a set of data blocks corresponding to the read offset; and

dispersed storage error decoding the decode threshold number of data blocks to reproduce the portion of the data object.

16. The non-transitory computer readable storage medium of claim 15, wherein the processing system determines to serially store the data object in response to receiving the data object in a store data request.

17. The non-transitory computer readable storage medium of claim 15, wherein facilitating storage of a first set of encoded data slices corresponding to a first IDA threshold number of received data blocks includes issuing write slice requests to the set of storage units for the first set of encoded data slices, and wherein facilitating storage of at least one subsequent set of encoded data slices corresponding to at least one subsequent IDA threshold number of received data blocks includes issuing slice append requests for the at least one subsequent set of encoded data slices.

18. The non-transitory computer readable storage medium of claim 15, wherein identifying the portion of the data object includes interpreting the retrieval request to identify at least one IDA threshold number of received data blocks for recovery.

19. The non-transitory computer readable storage medium of claim 18, wherein the read offset is generated based on an offset number of IDA threshold number of received data blocks from a first IDA threshold number of appended encoded data slices to the identified at least one IDA threshold number of received data blocks for recovery.

20. The non-transitory computer readable storage medium of claim 15, wherein retrieving the decode threshold number of data blocks includes issuing read slice requests that include the read offset to the set of storage units and receiving at least one of the decode threshold number of data blocks in response.