Storing a stream of data in a dispersed storage network

• US 9,537,609 B2
• Filed: 06/17/2013
• Issued: 01/03/2017
• Est. Priority Date: 08/02/2012
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Gladwin, S. Christopher, Markison, Timothy W., Dhuse, Greg, Shirley, Thomas Franklin Jr., Leggette, Wesley, Resch, Jason K., Grube, Gary W.
• Primary Examiner(s): Chan, Wing F
• Assistant Examiner(s): KRISHNAN, RAJESHWARI
• Application Number:
  US13/919,566
• Publication Number:
  US 20140040417A1
• Time in Patent Office: 1,296 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2053 where persistent mass storage functionality or persistent mass storage control functionality is redundant error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H03M 13/611 Specific encoding aspects, e.g. encoding by means of decoding
• CPC Class Codes: H04L 1/0076 Distributed coding, e.g. network coding, involving channel coding coding in both space and time H04L1/0618; cooperative diversity H04B7/022
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

A processing module of a computing device alternatingly sends a stream of data to a first or second processing device. When receiving the stream of data, the first processing device performs a first portion of a dispersed storage error encoding function on the received stream of data to produce a plurality of sets of a threshold number of slices and writes the plurality of sets of the threshold number of slices into first memory of a dispersed storage network (DSN). When not receiving the stream of data, the first processing device reads the plurality of sets of the threshold number of slices from the first memory, performs a second portion of the dispersed storage error encoding function using the plurality of sets of the threshold number of slices to produce a plurality of sets of redundancy slices, and writes the plurality of sets of redundancy slices into second DSN memory.

Claims

1. A method comprises:

performing, by at least one processing device, a first portion of a dispersed storage error encoding function on streaming data to produce a plurality of sets of a threshold number of encoded data slices, wherein the dispersed storage error encoding function includes parameters that define a total number of encoded data slices to be created per set and the threshold number as the minimum number of encoded data slices of a particular set to recover a corresponding portion of the streaming data, and wherein a difference between the total number and the threshold number corresponds to a redundancy number;

alternatingly, by the at least one processing device, establishing a write only mode for a first memory of a dispersed storage network (DSN) and a second memory of the DSN;

writing, by the at least one processing device, sets of the plurality of sets of the threshold number of encoded data slices to the first memory when the first memory is in the write only mode;

writing, by the at least one processing device, alternate sets of the plurality of sets of the threshold number of encoded data slices to the second memory when the second memory is in the write only mode;

when the first memory is not in the write only mode, reading, by a second processing device, the sets of the plurality of sets of the threshold number of encoded data slices from the first memory;

performing, by the second processing device, a second portion of the dispersed storage error encoding function on the read sets of the plurality of sets of the threshold number of encoded data slices to produce sets of redundancy encoded data slices; and

writing, by the second processing device, the sets of redundancy encoded data slices to a third memory of the DSN.

2. The method of claim 1 further comprises:

when the second memory is not in the write only mode, reading, by a third processing device, the alternate sets of the plurality of sets of the threshold number of encoded data slices from the second memory;

performing, by the third processing device, the second portion of the dispersed storage error encoding function on the read alternate sets of the plurality of sets of the threshold number of encoded data slices to produce alternate sets of redundancy encoded data slices; and

writing, by the third processing device, the alternate sets of redundancy encoded data slices to a fourth memory of the DSN.

3. The method of claim 1, wherein the performing a first portion of a dispersed storage error encoding function on the streaming data comprises:

performing, by a first processing device of the at least one processing device, the first portion of the dispersed storage error encoding function on a first alternating portion of the streaming data to produce the alternate sets of the plurality of sets of the threshold number of encoded data slices; and

performing, by the second processing device as another processing device of the at least one processing device, the first portion of the dispersed storage error encoding function on a second alternating portion of the streaming data to produce the sets of the plurality of sets of the threshold number of encoded data slices.

4. The method of claim 3 further comprises:

when the second memory is not in the write only mode, reading, by the first processing device, the alternate sets of the plurality of sets of the threshold number of encoded data slices from the second memory;

performing, by the first processing device, the second portion of the dispersed storage error encoding function on the read alternate sets of the plurality of sets of the threshold number of encoded data slices to produce alternate sets of redundancy encoded data slices; and

writing, by the first processing device, the alternate sets of redundancy encoded data slices to a fourth memory of the DSN.

5. The method of claim 3 further comprises:

alternatingly sending, by a processing module of a computing device, the streaming data to the first processing device or to the second processing device, wherein the alternatingly sending the streaming data includes;

establishing an alternating first time interval and second time interval;

sending the stream of data to the first processing device during the first time interval; and

sending the stream of data to the second processing device during the second time interval.

6. The method of claim 1 further comprises:

the first portion of the dispersed storage error encoding function corresponding to a unity matrix portion of an encoding matrix; and

the second portion of the dispersed storage error encoding function corresponding to a redundancy portion of the encoding matrix.

7. The method of claim 1 further comprises:

the first portion of the dispersed storage error encoding function corresponding to a level of encoding to produce the plurality of sets of the threshold number of encoded data slices; and

the second portion of the dispersed storage error encoding function corresponding to a level of encoding to produce the plurality of sets of the redundancy encoded data slices.

8. The method of claim 1 further comprises:

alternatingly, by the at least one processing device, establishing a write only mode for the first memory, the second memory, and a third memory of the DSN; and

writing, by the at least one processing device, further alternate sets of the plurality of sets of the threshold number of encoded data slices to the third memory when the third memory is in the write only mode.

9. The method of claim 1 further comprises:

generating, by a processing module of a computing device, address mapping of the streaming data to the first and second memories, wherein the address mapping links a corresponding one of the sets of the plurality of sets of the threshold number of encoded data slices with a corresponding one of the sets of the plurality of sets of redundancy encoded data slices.

10. The method of claim 1 further comprises:

subsequent to receiving the streaming data, transferring the plurality of sets of the threshold number of encoded data slices from the first memory to the third memory.