Storing data integrity information utilizing dispersed storage

• US 9,098,441 B2
• Filed: 07/09/2013
• Issued: 08/04/2015
• Est. Priority Date: 11/01/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K.
• Primary Examiner(s): Armouche, Hadi
• Assistant Examiner(s): HO, DAO Q
• Application Number:
  US13/937,400
• Publication Number:
  US 20130297947A1
• Time in Patent Office: 756 Days
• Field of Search: 713/189
• US Class Current: 1/1
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: H04L 41/06 Management of faults, events, alarms or notifications
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module generating an integrity check value for each encoded data slice of a set of encoded data slices to produce a set of integrity check values. The method continues with the processing module encoding the set of integrity check values to produce encoded integrity check values. The method continues with the processing module sending the encoded integrity check values for storage in a memory system.

Claims

1. A method comprises:

obtaining at least a decode threshold number of encoded data slices of a set of encoded data slices that are stored in a first set of dispersed storage (DS) units in a memory system;

obtaining, separately from the obtaining the encoded data slices, encoded integrity check values that are stored in a second set of dispersed storage (DS) units in the memory system;

decoding the encoded integrity check values to produce a set of stored integrity check values, wherein each integrity check value of the set of stored integrity check values is associated with a corresponding encoded data slice of the set of encoded data slices, wherein the set of stored integrity check values were encoded using encryption or a dispersed storage error encoding function to produce the encoded integrity check values;

calculating an integrity check value for each of the at least the decode threshold number of encoded data slices to produce a set of calculated integrity check values;

for each the at least the decode threshold number of encoded data slices, comparing a corresponding integrity check value of the set of stored integrity check values with a corresponding integrity check value of the set of calculated integrity check values; and

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is favorable, indicating that the at least the decode threshold number of encoded data slices is valid.

2. The method of claim 1, wherein the decoding the encoded integrity check values comprises at least one of:

decrypting the encoded integrity check values to produce decrypted integrity check values; and

dispersed storage error decoding the encoded integrity check values or the decrypted integrity check values to produce dispersed storage error decoded integrity check values, wherein the set of stored integrity check values includes the decrypted integrity check values or the dispersed storage error decoded integrity check values.

3. The method of claim 2, wherein the decrypting the encrypted integrity check values comprises at least one of:

decrypting the integrity check values utilizing an all or nothing (AONT) transformation;

decrypting the integrity check values utilizing a public-key;

decrypting the integrity check values utilizing a private key; and

verifying a signature of the integrity check values utilizing the private key.

4. The method of claim 1 further comprises:

when the at least the decode threshold number of encoded data slices is valid, dispersed storage error decoding the at least the decode threshold number of encoded data slices to reproduce a data segment.

5. The method of claim 1 further comprises:

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is unfavorable, indicating that the at least the decode threshold number of encoded data slices is invalid.

6. The method of claim 1, wherein the calculating the integrity check value for each of the at least the decode threshold number of encoded data slices comprises:

applying an integrity check function on the encoded data slice to generate the integrity check value, wherein the integrity check function includes at least one of a hashing function, a cyclic redundancy check function, a parity check function, and a mask generating function (MGF).

7. A computer comprises:

an interface;

a memory; and

a processing module operable to;

obtain at least a decode threshold number of encoded data slices of a set of encoded data slices that are stored in a first set of dispersed storage (DS) units in a memory system;

obtain, separately from the obtaining the encoded data slices, encoded integrity check values that are stored in a second set of dispersed storage (DS) units in the memory system;

decode the encoded integrity check values to produce a set of stored integrity check values, wherein each integrity check value of the set of stored integrity check values is associated with a corresponding encoded data slice of the set of encoded data slices, wherein the set of stored integrity check values were encoded using encryption or a dispersed storage error encoding function to produce the encoded integrity check values;

calculate an integrity check value for each of the at least the decode threshold number of encoded data slices to produce a set of calculated integrity check values;

for each the at least the decode threshold number of encoded data slices, compare a corresponding integrity check value of the set of stored integrity check values with a corresponding integrity check value of the set of calculated integrity check values; and

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is favorable, indicate that the at least the decode threshold number of encoded data slices is valid.

8. The computer of claim 7, wherein the processing module functions to decode the encoded integrity check values by at least one of:

decrypting the encoded integrity check values to produce decrypted integrity check values; and

dispersed storage error decoding the encoded integrity check values or the decrypted integrity check values to produce dispersed storage error decoded integrity check values, wherein the set of stored integrity check values includes the decrypted integrity check values or the dispersed storage error decoded integrity check values.

9. The computer of claim 8, wherein the processing module functions to decrypt the encrypted integrity check values by at least one of:

decrypting the integrity check values utilizing an all or nothing (AONT) transformation;

decrypting the integrity check values utilizing a public-key;

decrypting the integrity check values utilizing a private key; and

verifying a signature of the integrity check values utilizing the private key.

10. The computer of claim 7, wherein the processing module further functions to:

when the at least the decode threshold number of encoded data slices is valid, dispersed storage error decode the at least the decode threshold number of encoded data slices to reproduce a data segment.

11. The computer of claim 7, wherein the processing module further functions to:

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is unfavorable, indicate that the at least the decode threshold number of encoded data slices is invalid.

12. The computer of claim 7, wherein the processing module functions to calculate the integrity check value for each of the at least the decode threshold number of encoded data slices by:

applying an integrity check function on the encoded data slice to generate the integrity check value, wherein the integrity check function includes at least one of a hashing function, a cyclic redundancy check function, a parity check function, and a mask generating function (MGF).