STORING DATA INTEGRITY INFORMATION UTILIZING DISPERSED STORAGE

• US 20120110346A1
• Filed: 10/11/2011
• Published: 05/03/2012
• Est. Priority Date: 11/01/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K.
• Granted Patent: US 9,274,977 B2
• Time in Patent Office: Days
• US Class Current: 713/189
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: H04L 41/06 Management of faults, events, alarms or notifications
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module generating an integrity check value for each encoded data slice of a set of encoded data slices to produce a set of integrity check values. The method continues with the processing module encoding the set of integrity check values to produce encoded integrity check values. The method continues with the processing module sending the encoded integrity check values for storage in a memory system.

Claims

1. A method comprises:

generating an integrity check value for each encoded data slice of a set of encoded data slices to produce a set of integrity check values;

encoding the set of integrity check values to produce encoded integrity check values; and

sending the encoded integrity check values for storage in a memory system.

2. The method of claim 1, wherein the encoding comprises at least one of:

encrypting the set of integrity check values to produce encrypted integrity check information; and

dispersed storage error encoding the set of integrity check values or the encrypted integrity check information to produce a set of encoded integrity check slices, wherein the encoded integrity check values includes the set of encoded integrity check slices or the encrypted integrity check information.

3. The method of claim 2 further comprises:

for the each encoded data slice of the set of encoded data slices;

sending the encoded data slice to a dispersed storage (DS) unit of a set of DS units for storage therein; and

sending one of the set of encoded integrity check slices to the DS unit for storage therein.

4. The method of claim 2 further comprises:

sending the set of encoded data slices to a first set of dispersed storage (DS) units; and

sending the set of encoded integrity check slices to a second set of DS units.

5. The method of claim 2, wherein the encrypting the set of integrity check values comprises at least one of:

encrypting the set of integrity check values utilizing an all or nothing (AONT) transformation;

encrypting the set of integrity check values utilizing a public-key;

encrypting the set of integrity check values utilizing a private key; and

signing the set of integrity check values utilizing the private-key.

6. The method of claim 1 further comprises:

storing the encoded integrity check values in a local memory.

7. The method of claim 1, wherein the generating the integrity check value comprises:

applying an integrity check function on the encoded data slice to generate the integrity check value, wherein the integrity check function includes at least one of a hashing function, a cyclic redundancy check function, a parity check function, and a mask generating function (MGF).

8. A method comprises:

obtaining at least a decode threshold number of encoded data slices of a set of encoded data slices;

obtaining encoded integrity check values;

decoding the encoded integrity check values to produce a set of stored integrity check values, wherein each integrity check value of the set of stored integrity check values is associated with a corresponding encoded data slice of the set of encoded data slices;

calculating an integrity check value for each of the at least the decode threshold number of encoded data slices to produce a set of calculated integrity check values;

for each the at least the decode threshold number of encoded data slices, comparing a corresponding integrity check value of the set of stored integrity check values with a corresponding integrity check value of the set of calculated integrity check values;

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is favorable, indicating that the at least the decode threshold number of encoded data slices is valid.

9. The method of claim 8, wherein the decoding the encoded integrity check values comprises at least one of:

decrypting the encoded integrity check values to produce decrypted integrity check values; and

dispersed storage error decoding the encoded integrity check values or the decrypted integrity check values to produce dispersed storage error decoded integrity check values,

wherein the set of stored integrity check values includes the decrypted integrity check values or the dispersed storage error decoded integrity check values.

10. The method of claim 9, wherein the decrypting the encrypted integrity check values comprises at least one of:

decrypting the integrity check values utilizing an all or nothing (AONT) transformation;

decrypting the integrity check values utilizing a public-key;

decrypting the integrity check values utilizing a private key; and

verifying a signature of the integrity check values utilizing the private key.

11. The method of claim 8 further comprises:

when the at least the decode threshold number of encoded data slices is valid, dispersed storage error decoding the at least the decode threshold number of encoded data slices to reproduce a data segment.

12. The method of claim 8 further comprises:

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is unfavorable, indicating that the at least the decode threshold number of encoded data slices is invalid.

13. The method of claim 8, wherein the calculating the integrity check value for each of the at least the decode threshold number of encoded data slices comprises:

applying an integrity check function on the encoded data slice to generate the integrity check value, wherein the integrity check function includes at least one of a hashing function, a cyclic redundancy check function, a parity check function, and a mask generating function (MGF).

14. A computer comprises:

an interface;

a memory; and

a processing module operable to;

generate an integrity check value for each encoded data slice of a set of encoded data slices to produce a set of integrity check values;

encode the set of integrity check values to produce encoded integrity check values; and

send, via the interface, the encoded integrity check values for storage in a memory system.

15. The computer of claim 14, wherein the processing module functions to encode by at least one of:

encrypting the set of integrity check values to produce encrypted integrity check information; and

dispersed storage error encoding the set of integrity check values or the encrypted integrity check information to produce a set of encoded integrity check slices, wherein the encoded integrity check values includes the set of encoded integrity check slices or the encrypted integrity check information.

16. The computer of claim 15, wherein the processing module further functions to:

for the each encoded data slice of the set of encoded data slices;

send, via the interface, the encoded data slice to a dispersed storage (DS) unit of a set of DS units for storage therein; and

send, via the interface, one of the set of encoded integrity check slices to the DS unit for storage therein.

17. The computer of claim 15, wherein the processing module further functions to:

send, via the interface, the set of encoded data slices to a first set of dispersed storage (DS) units; and

send, via the interface, the set of encoded integrity check slices to a second set of DS units.

18. The computer of claim 15, wherein the processing module functions to encrypt the set of integrity check values by at least one of:

encrypting the set of integrity check values utilizing an all or nothing (AONT) transformation;

encrypting the set of integrity check values utilizing a public-key;

encrypting the set of integrity check values utilizing a private key; and

signing the set of integrity check values utilizing the private-key.

19. The computer of claim 14, wherein the processing module further functions to:

store the encoded integrity check values in the memory.

20. The computer of claim 14, wherein the processing module functions to generate the integrity check value by:

applying an integrity check function on the encoded data slice to generate the integrity check value, wherein the integrity check function includes at least one of a hashing function, a cyclic redundancy check function, a parity check function, and a mask generating function (MGF).

21. A computer comprises:

an interface;

a memory; and

a processing module operable to;

obtain at least a decode threshold number of encoded data slices of a set of encoded data slices;

obtain encoded integrity check values;

decode the encoded integrity check values to produce a set of stored integrity check values, wherein each integrity check value of the set of stored integrity check values is associated with a corresponding encoded data slice of the set of encoded data slices;

calculate an integrity check value for each of the at least the decode threshold number of encoded data slices to produce a set of calculated integrity check values;

for each the at least the decode threshold number of encoded data slices, compare a corresponding integrity check value of the set of stored integrity check values with a corresponding integrity check value of the set of calculated integrity check values;

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is favorable, indicate that the at least the decode threshold number of encoded data slices is valid.

22. The computer of claim 21, wherein the processing module functions to decode the encoded integrity check values by at least one of:

decrypting the encoded integrity check values to produce decrypted integrity check values; and

dispersed storage error decoding the encoded integrity check values or the decrypted integrity check values to produce dispersed storage error decoded integrity check values, wherein the set of stored integrity check values includes the decrypted integrity check values or the dispersed storage error decoded integrity check values.

23. The computer of claim 22, wherein the processing module functions to decrypt the encrypted integrity check values by at least one of:

decrypting the integrity check values utilizing an all or nothing (AONT) transformation;

decrypting the integrity check values utilizing a public-key;

decrypting the integrity check values utilizing a private key;

verifying a signature of the integrity check values utilizing the private key.

24. The computer of claim 21, wherein the processing module further functions to:

when the at least the decode threshold number of encoded data slices is valid, dispersed storage error decode the at least the decode threshold number of encoded data slices to reproduce a data segment.

25. The computer of claim 21, wherein the processing module further functions to:

when the comparing of the corresponding integrity check values for each of the at least the decode threshold number of encoded data slices is unfavorable, indicate that the at least the decode threshold number of encoded data slices is invalid.

26. The computer of claim 21, wherein the processing module functions to calculate the integrity check value for each of the at least the decode threshold number of encoded data slices by:

applying an integrity check function on the encoded data slice to generate the integrity check value, wherein the integrity check function includes at least one of a hashing function, a cyclic redundancy check function, a parity check function, and a mask generating function (MGF).