DETECTING INTENTIONAL CORRUPTION OF DATA IN A DISPERSED STORAGE NETWORK

• US 20140365831A1
• Filed: 07/14/2014
• Published: 12/11/2014
• Est. Priority Date: 08/17/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Dhuse, Greg, Leggette, Wesley, Baptist, Andrew
• Granted Patent: US 9,087,016 B2
• Time in Patent Office: Days
• US Class Current: 714/42
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1612 where the redundant component is persistent storage
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 2209/16 Obfuscation or hiding, e.g. involving white box
• CPC Class Codes: H04L 9/00 Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
• CPC Class Codes: H04L 9/085 Secret sharing or secret splitting, e.g. threshold schemes
• CPC Class Codes: H04L 9/0863 involving passwords or one-time passwords network architectures or network communication protocols for using one-time keys in a packet data network H04L63/067
• CPC Class Codes: H04L 9/0869 involving random numbers or seeds
• CPC Class Codes: H04L 9/0877 using additional device, e.g. trusted platform module [TPM], smartcard, USB or hardware security module [HSM]
• CPC Class Codes: H04L 9/0894 Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
• CPC Class Codes: H04L 9/32 including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
• CPC Class Codes: H04L 9/321 involving a third party or a trusted authority
• CPC Class Codes: H04L 9/3263 involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements network architectures or network communication protocols for supporting authentication of entities using certificates in a packet data network H04L63/0823
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A method begins by a dispersed storage (DS) processing module of a DS unit selecting a data slice for corruption analysis and requesting integrity information for the data slice from one or more other DS units of a dispersed storage network. When the one or more requested integrity information is received, the method continues with the DS processing module analyzing the one or more received integrity information and local integrity information of the data slice stored in the DS unit. When the analysis of the one or more received integrity information and the local integrity information of the data slice is unfavorable, the method continues with the DS processing module identifying the data slice as being corrupted.

Claims

1. A method for detecting intentional corruption of data in a dispersed storage network (DSN), the method comprises:

selecting, by a storage unit of the DSN, a selected encoded data slice of a set of encoded data slices for corruption analysis, wherein a data segment of data was dispersed error encoded to produce the set of encoded data slices, wherein an integrity values was created for each encoded data slice of at least some of the set of encoded data slices to produce a set of integrity values, wherein encoded data slices of the set of encoded data slices and the set of integrity values are stored in a set of storage units, and wherein the set of storage units includes the storage unit;

requesting and received, by the storage unit, the set of integrity values from another storage unit of the set of storage units;

comparing, by the storage unit, the received set of integrity values with the set of integrity values that stored by the storage unit; and

when the comparison of the received set of integrity values with the set of integrity values that stored by the storage unit is unfavorable, identifying, by the storage unit, the selected encoded data slice as being corrupted.

2. The method of claim 1, wherein the comparing comprises:

extracting the integrity value of the selected encoded data slice from the received set of encoded data slices to produce a received integrity value;

extracting the integrity value of the selected encoded data slice from the set of encoded data slices stored by the storage unit to produce a local integrity value; and

comparing the received integrity value with the local integrity value.

3. The method of claim 1, wherein the comparing comprises:

extracting the integrity value for a threshold number of encoded data slices from the received set of encoded data slices to produce a received threshold set of integrity values;

extracting the integrity for the threshold number of encoded data slices from the set of encoded data slices stored by the storage unit to produce a local set of integrity values; and

comparing the received set of integrity values with the local set of integrity values.

4. The method of claim 1 further comprises:

requesting and received, by the storage unit, the set of integrity values from a second other storage unit of the set of storage units;

comparing, by the storage unit, the received set of integrity values from the second other storage unit with the set of integrity values that stored by the storage unit;

when the comparison of the received set of integrity values from the second other storage unit with the set of integrity values that stored by the storage unit is unfavorable, identifying, by the storage unit, the selected encoded data slice as being corrupted; and

when the comparison of both the received set of integrity values with the set of integrity values that stored by the storage unit and the received set of integrity values from the second other storage unit with the set of integrity values that stored by the storage unit are favorable, indicating, by the storage unit, that the selected encoded data slice is not corrupted.

5. The method of claim 1 further comprises:

initiating a rebuilding process to rebuild the selected encoded data slice when the selected encoded data slice is corrupted.

6. The method of claim 5, wherein the rebuilding process comprises one of:

a partial rebuilding process in which the identified data slice is rebuilt; and

a rebuilding process in which a data segment is rebuilt from a set of data slices and the data segment is re-encoded to produce a rebuilt data slice.

7. A computer readable storage device comprises:

a first memory section that stores operational instructions that, when executed by a storage unit, causes the storage unit to;

select a selected encoded data slice of a set of encoded data slices for corruption analysis, wherein a data segment of data was dispersed error encoded to produce the set of encoded data slices, wherein an integrity values was created for each encoded data slice of at least some of the set of encoded data slices to produce a set of integrity values, wherein encoded data slices of the set of encoded data slices and the set of integrity values are stored in a set of storage units, and wherein the set of storage units includes the storage unit;

a second memory section that stores operational instructions that, when executed by the storage unit, causes the storage unit to;

request and receive the set of integrity values from another storage unit of the set of storage units; and

a third memory section that stores operational instructions that, when executed by the storage unit, causes the storage unit to;

compare the received set of integrity values with the set of integrity values that stored by the storage unit; and

when the comparison of the received set of integrity values with the set of integrity values that stored by the storage unit is unfavorable, identify the selected encoded data slice as being corrupted.

8. The computer readable storage device of claim 7, wherein the third memory section further stores operational instructions that, when executed by the storage unit, causes the storage unit to perform the comparing by:

extracting the integrity value of the selected encoded data slice from the received set of encoded data slices to produce a received integrity value;

extracting the integrity value of the selected encoded data slice from the set of encoded data slices stored by the storage unit to produce a local integrity value; and

comparing the received integrity value with the local integrity value.

9. The computer readable storage device of claim 7, wherein the third memory section further stores operational instructions that, when executed by the storage unit, causes the storage unit to perform the comparing by:

extracting the integrity value for a threshold number of encoded data slices from the received set of encoded data slices to produce a received threshold set of integrity values;

extracting the integrity for the threshold number of encoded data slices from the set of encoded data slices stored by the storage unit to produce a local set of integrity values; and

comparing the received set of integrity values with the local set of integrity values.

10. The computer readable storage device of claim 7 further comprises:

the second memory section further storing operational instructions that, when executed by the storage unit, causes the storage unit to;

request and receive the set of integrity values from a second other storage unit of the set of storage units;

the third memory section further storing operational instructions that, when executed by the storage unit, causes the storage unit to;

compare the received set of integrity values from the second other storage unit with the set of integrity values that stored by the storage unit;

when the comparison of the received set of integrity values from the second other storage unit with the set of integrity values that stored by the storage unit is unfavorable, identify the selected encoded data slice as being corrupted; and

when the comparison of both the received set of integrity values with the set of integrity values that stored by the storage unit and the received set of integrity values from the second other storage unit with the set of integrity values that stored by the storage unit are favorable, indicate that the selected encoded data slice is not corrupted.

11. The computer readable storage device of claim 7, wherein the third memory section further stores operational instructions that, when executed by the storage unit, causes the storage unit to:

initiate a rebuilding process to rebuild the selected encoded data slice when the selected encoded data slice is corrupted.

12. The computer readable storage device of claim 11, wherein the rebuilding process comprises one of:

a partial rebuilding process in which the identified data slice is rebuilt; and

a rebuilding process in which a data segment is rebuilt from a set of data slices and the data segment is re-encoded to produce a rebuilt data slice.

13. A storage unit comprises:

an interface;

memory; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

select a selected encoded data slice of a set of encoded data slices for corruption analysis, wherein a data segment of data was dispersed error encoded to produce the set of encoded data slices, wherein an integrity values was created for each encoded data slice of at least some of the set of encoded data slices to produce a set of integrity values, wherein encoded data slices of the set of encoded data slices and the set of integrity values are stored in a set of storage units, and wherein the set of storage units includes the storage unit;

request and receive, via the interface, the set of integrity values from another storage unit of the set of storage units; and

compare the received set of integrity values with the set of integrity values that stored in the memory; and

when the comparison of the received set of integrity values with the set of integrity values that stored in the memory is unfavorable, identify the selected encoded data slice as being corrupted.

14. The storage unit of claim 13, wherein the processing module is further operable to perform the comparing by:

extracting the integrity value of the selected encoded data slice from the received set of encoded data slices to produce a received integrity value;

extracting the integrity value of the selected encoded data slice from the set of encoded data slices stored by the storage unit to produce a local integrity value; and

comparing the received integrity value with the local integrity value.

15. The storage unit of claim 13, wherein the processing module is further operable to perform the comparing by:

extracting the integrity value for a threshold number of encoded data slices from the received set of encoded data slices to produce a received threshold set of integrity values;

extracting the integrity for the threshold number of encoded data slices from the set of encoded data slices stored by the storage unit to produce a local set of integrity values; and

comparing the received set of integrity values with the local set of integrity values.

16. The storage unit of claim 13, wherein the processing module is further operable to:

request and receive, via the interface, the set of integrity values from a second other storage unit of the set of storage units;

compare the received set of integrity values from the second other storage unit with the set of integrity values that stored in the memory;

when the comparison of the received set of integrity values from the second other storage unit with the set of integrity values that stored in the memory is unfavorable, identify the selected encoded data slice as being corrupted; and

when the comparison of both the received set of integrity values with the set of integrity values that stored in the memory and the received set of integrity values from the second other storage unit with the set of integrity values that stored in the memory are favorable, indicate that the selected encoded data slice is not corrupted.

17. The storage unit of claim 13, wherein the processing module is further operable to:

initiate a rebuilding process to rebuild the selected encoded data slice when the selected encoded data slice is corrupted.

18. The storage unit of claim 17, wherein the rebuilding process comprises one of:

a partial rebuilding process in which the identified data slice is rebuilt; and

a rebuilding process in which a data segment is rebuilt from a set of data slices and the data segment is re-encoded to produce a rebuilt data slice.