Storing a stream of data in a dispersed storage network

• US 10,200,156 B2
• Filed: 07/18/2016
• Issued: 02/05/2019
• Est. Priority Date: 08/02/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Gladwin, S. Christopher, Markison, Timothy W., Dhuse, Greg, Shirley, Jr., Thomas Franklin, Leggette, Wesley, Resch, Jason K., Grube, Gary W.
• Primary Examiner(s): Chan, Wing F
• Assistant Examiner(s): Krishnan, Rajeshwari
• Application Number:
  US15/213,329
• Publication Number:
  US 20160328294A1
• Time in Patent Office: 932 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2053 where persistent mass storage functionality or persistent mass storage control functionality is redundant error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H03M 13/611 Specific encoding aspects, e.g. encoding by means of decoding
• CPC Class Codes: H04L 1/0076 Distributed coding, e.g. network coding, involving channel coding coding in both space and time H04L1/0618; cooperative diversity H04B7/022
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A processing module of a computing device alternatingly sends a stream of data to a first or second processing device. When receiving the stream of data, the first processing device performs a first portion of a dispersed storage error encoding function on the received stream of data to produce a plurality of sets of a threshold number of slices and writes the plurality of sets of the threshold number of slices into first memory of a dispersed storage network (DSN). When not receiving the stream of data, the first processing device reads the plurality of sets of the threshold number of slices from the first memory, performs a second portion of the dispersed storage error encoding function using the plurality of sets of the threshold number of slices to produce a plurality of sets of redundancy slices, and writes the plurality of sets of redundancy slices into second DSN memory.

Claims

1. A dispersed storage system comprises:

at least one first computing device of a dispersed storage network (DSN), executing a first module that causes the at least one first computing device to;

perform a first portion of a dispersed storage error encoding function on streaming data to produce a plurality of sets of a threshold number of encoded data slices;

alternatingly establish a write only mode for a first memory of DSN and a second memory of the DSN;

write sets of the plurality of sets of the threshold number of encoded data slices to the first memory when the first memory is in the write only mode; and

write alternate sets of the plurality of sets of the threshold number of encoded data slices to the second memory when the second memory is in the write only mode; and

a second computing device of the DSN, executing a second module that causes the second computing device to;

when the first memory is not in the write only mode, read the sets of the plurality of sets of the threshold number of encoded data slices from the first memory;

perform a second portion of the dispersed storage error encoding function on the read sets of the plurality of sets of the threshold number of encoded data slices to produce sets of redundancy encoded data slices; and

write the sets of redundancy encoded data slices to a third memory of the DSN.

2. The dispersed storage system of claim 1 further comprises:

a third module, when operable in a third computing device of the DSN or the second computing device, causes the second or third computing device to;

when the second memory is not in the write only mode, read the alternate sets of the plurality of sets of the threshold number of encoded data slices from the second memory;

perform the second portion of the dispersed storage error encoding function on the read alternate sets of the plurality of sets of the threshold number of encoded data slices to produce alternate sets of redundancy encoded data slices; and

write the alternate sets of redundancy encoded data slices to a fourth memory of the DSN.

3. The dispersed storage system of claim 1, wherein the first module further comprises:

a first sub-module, when operable in one of the at least one first computing device, causes the one of the at least one first computing device to perform the first portion of the dispersed storage error encoding function on a first alternating portion of the streaming data to produce the alternate sets of the plurality of sets of the threshold number of encoded data slices; and

a second sub-module, when operable in another one of the at least one first computing device, causes the other one of the at least one first computing device to perform the first portion of the dispersed storage error encoding function on a second alternating portion of the streaming data to produce the sets of the plurality of sets of the threshold number of encoded data slices.

4. The dispersed storage system of claim 3, wherein:

when the second memory is not in the write only mode, the first sub-module, when operable in the one of the at least one first computing device, further causes the one of the at least one first computing device to;

read the alternate sets of the plurality of sets of the threshold number of encoded data slices from the second memory;

perform the second portion of the dispersed storage error encoding function on the read alternate sets of the plurality of sets of the threshold number of encoded data slices to produce alternate sets of redundancy encoded data slices; and

write the alternate sets of redundancy encoded data slices to a fourth memory of the DSN.

5. The dispersed storage system of claim 3 further comprises:

a source computing device that alternatingly sends the streaming data to the one of the at least one first computing device or to the other one of the at least one first computing device, wherein the alternatingly sending the streaming data includes;

establishing an alternating first time interval and second time interval;

sending the stream of data to the one of the at least one first computing device during the first time interval; and

sending the stream of data to the other one of the at least one first computing device during the second time interval.

6. The dispersed storage system of claim 1 further comprises:

the first portion of the dispersed storage error encoding function corresponding to a unity matrix portion of an encoding matrix; and

the second portion of the dispersed storage error encoding function corresponding to a redundancy portion of the encoding matrix.

7. The dispersed storage system of claim 1 further comprises:

the first portion of the dispersed storage error encoding function corresponding to a level of encoding to produce the plurality of sets of the threshold number of encoded data slices; and

the second portion of the dispersed storage error encoding function corresponding to a level of encoding to produce the plurality of sets of the redundancy encoded data slices.

8. The dispersed storage system of claim 1 wherein:

the at least one first computing device, executing the first module that further causes the at least one first computing device to;

alternatingly establish a write only mode for the first memory, the second memory, and a third memory of the DSN; and

write sets of the plurality of sets of the threshold number of encoded data slices to the third memory when the third memory is in the write only mode.

9. The dispersed storage system of claim 1 further comprises:

a source computing device that generates an address mapping that links a corresponding one of the sets of the plurality of sets of the threshold number of encoded data slices with a corresponding one of the sets of the plurality of sets of redundancy encoded data slices.

10. The dispersed storage system of claim 1 wherein:

the at least one first computing device, executing the first module that further causes the at least one first computing device to;

transfer the plurality of sets of the threshold number of encoded data slices from the first memory to the third memory.