Encoding data utilizing a zero information gain function

• US 8,677,214 B2
• Filed: 09/12/2012
• Issued: 03/18/2014
• Est. Priority Date: 10/04/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): Decady, Albert
• Assistant Examiner(s): ALSHACK, OSMAN M
• Application Number:
  US13/611,962
• Publication Number:
  US 20130086450A1
• Time in Patent Office: 552 Days
• Field of Search: 714/763, 714/758
• US Class Current: 714/763
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries
• CPC Class Codes: H03M 11/08 by operating selected combinations of multifunction keys
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/15 Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes H03M13/17 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/152 Bose-Chaudhuri-Hocquenghem [BCH] codes
• CPC Class Codes: H03M 13/19 Single error correction without using particular properties of the cyclic codes, e.g. Hamming codes, extended or generalised Hamming codes
• CPC Class Codes: H03M 13/23 using convolutional codes, e.g. unit memory codes
• CPC Class Codes: H03M 13/2906 using block codes H03M13/2957 takes precedence
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H03M 13/616 Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module encoding data using a dispersed storage error coding function to produce a set of encoded data slices. The method continues with the DS processing module encoding a first encoded data slice of the set of encoded data slices using a zero information gain (ZIG) function based on a second encoded data slice of the set of encoded data slices to produce a ZIG encoded data slice. The method continues with the DS processing module outputting the ZIG encoded data slice and a subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than a decode threshold number of encoded data slices and does not include the first or the second encoded data slice.

Claims

1. A method comprises:

encoding data using a dispersed storage error coding function to produce a set of encoded data slices, wherein the data is re-creatable from a decode threshold number of encoded data slices of the set of encoded data slices;

generating a zero information gain (ZIG) encoded data slice that represents a component of recovery information of a first encoded data slice of the set of encoded data slices using a ZIG function and a second encoded data slice of the set of encoded data slices; and

outputting the ZIG encoded data slice and a subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than the decode threshold number of encoded data slices and does not include the first or the second encoded data slice.

2. The method of claim 1, wherein the encoding the data comprises:

dividing the data into data blocks;

encoding the data blocks using an encoding matrix of the dispersed storage error coding function to produce encoded data blocks; and

generating the set of encoded data slices from the encoded data blocks.

3. The method of claim 1, wherein the generating the ZIG encoded data slice comprises:

generating a decoding matrix for the first encoded data slice based on an encoding matrix of the dispersed storage error coding function; and

generating the ZIG encoded data slice based on the decoding matrix and on the second encoded data slice.

4. The method of claim 3, wherein the generating the decoding matrix comprises:

reducing the encoding matrix to produce a square matrix based on the first encoded data slice and the subset of encoded data slices; and

inverting the square matrix to produce the decoding matrix.

5. The method of claim 3, wherein the generating the ZIG encoded data slice based on the decoding matrix and on the second encoded data slice comprises:

encoding the second encoded data slice using the decoding matrix to produce a vector; and

encoding the vector by a row of the decoding matrix corresponding to the first encoded data slice to produce the ZIG encoded data slice.

6. The method of claim 1, wherein the outputting the ZIG encoded data slice and the subset of encoded data slices comprises at least one of:

outputting to a receiving entity in support of a communication with the receiving entity;

outputting to a dispersed storage network (DSN) memory for storage therein; and

outputting to at least one intermediary device.

7. The method of claim 1 further comprises:

generating a second ZIG encoded data slice that represents a component of recovery information of a third encoded data slice of the set of encoded data slices using the ZIG function and the second encoded data slice or a fourth encoded data slice of the set of encoded data slices; and

outputting the ZIG encoded data slice, the second ZIG encoded data slice, and the subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than the decode threshold number of encoded data slices and does not include the first, second, or third encoded data slice and, when the third encoded data slice is encoded based on the fourth encoded data slice, the subset of encoded data slices further does not include the fourth encoded data slice.

8. The method of claim 1 further comprises:

generating a second ZIG encoded data slice that represents a second component of the recovery information of the first encoded data slice of the set of encoded data slices using the ZIG function and a third encoded data slice of the set of encoded data slices;

combining the second ZIG encoded data slice with the ZIG encode data slice to produce the ZIG encoded data slice; and

outputting the ZIG encoded data slice and the subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than the decode threshold number of encoded data slices and does not include the first, second, or third encoded data slice.

9. The method of claim 1, wherein the outputting further comprises:

obfuscating the ZIG encoded data slice utilizing an obfuscation function prior to outputting the ZIG encoded data slice.

10. The method of claim 1 further comprises:

outputting ZIG function information regarding the encoding of the first encoded data slice.

11. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

encode data using a dispersed storage error coding function to produce a set of encoded data slices, wherein the data is re-creatable from a decode threshold number of encoded data slices of the set of encoded data slices;

a second module, when operable within the computing device, causes the computing device to;

generate a zero information gain (ZIG) encoded data slice that represents a component of recovery information of a first encoded data slice of the set of encoded data slices using a ZIG function and a second encoded data slice of the set of encoded data slices; and

a third module, when operable within the computing device, causes the computing device to;

output the ZIG encoded data slice and a subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than the decode threshold number of encoded data slices and does not include the first or the second encoded data slice.

12. The DS module of claim 11, wherein the first module functions to encode the data by:

dividing the data into data blocks;

encoding the data blocks using an encoding matrix of the dispersed storage error coding function to produce encoded data blocks; and

generating the set of encoded data slices from the encoded data blocks.

13. The DS module of claim 11, wherein the second module functions to generate the ZIG encoded data slice by:

generating a decoding matrix for the first encoded data slice based on an encoding matrix of the dispersed storage error coding function; and

generating the ZIG encoded data slice based on the decoding matrix and on the second encoded data slice.

14. The DS module of claim 13, wherein the second module functions to generate the decoding matrix by:

reducing the encoding matrix to produce a square matrix based on the first encoded data slice and the subset of encoded data slices; and

inverting the square matrix to produce the decoding matrix.

15. The DS module of claim 13, wherein the second module functions to generate the ZIG encoded data slice based on the decoding matrix and on the second encoded data slice by:

encoding the second encoded data slice using the decoding matrix to produce a vector; and

encoding the vector by a row of the decoding matrix corresponding to the first encoded data slice to produce the ZIG encoded data slice.

16. The DS module of claim 11, wherein the third module functions to output the ZIG encoded data slice and the subset of encoded data slices by at least one of:

outputting to a receiving entity in support of a communication with the receiving entity;

outputting to a dispersed storage network (DSN) memory for storage therein; and

outputting to at least one intermediary device.

17. The DS module of claim 11 further comprises:

the second module further functions to generate a second ZIG encoded data slice that represents a component of recovery information of a third encoded data slice of the set of encoded data slices using the ZIG function and the second encoded data slice or a fourth encoded data slice of the set of encoded data slices; and

the third module further functions to output the ZIG encoded data slice, the second ZIG encoded data slice, and the subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than the decode threshold number of encoded data slices and does not include the first, second, or third encoded data slice and, when the third encoded data slice is encoded based on the fourth encoded data slice, the subset of encoded data slices further does not include the fourth encoded data slice.

18. The DS module of claim 11 further comprises:

the second module further functions to;

generate a second ZIG encoded data slice that represents a second component of the recovery information of the first encoded data slice of the set of encoded data slices using the ZIG function and a third encoded data slice of the set of encoded data slices; and

combine the second ZIG encoded data slice with the ZIG encode data slice to produce the ZIG encoded data slice; and

the third module further functions to output the ZIG encoded data slice and the subset of encoded data slices of the set of encoded data slices, wherein the subset of encoded data slices includes less than the decode threshold number of encoded data slices and does not include the first, second, or third encoded data slice.

19. The DS module of claim 11, wherein the third module further functions to output by:

obfuscating the ZIG encoded data slice utilizing an obfuscation function prior to outputting the ZIG encoded data slice.

20. The DS module of claim 11 further comprises:

the third module further functions to output ZIG function information regarding the encoding of the first encoded data slice.