IDENTIFYING A POTENTIALLY COMPROMISED ENCODED DATA SLICE

• US 20130198588A1
• Filed: 12/18/2012
• Published: 08/01/2013
• Est. Priority Date: 01/31/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: RESCH, JASON K., LEGGETTE, WESLEY, MOTWANI, MANISH
• Granted Patent: US 8,990,664 B2
• Time in Patent Office: Days
• US Class Current: 714/763
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1096 Parity calculation or recalculation after configuration or reconfiguration of the system
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module selecting a data segment and verifying integrity values of encoded data slices generated by encoding the data segment. When integrity values of a decode threshold number of encoded data slices are affirmatively verified, the method continues with the DS processing module verifying an integrity value of the data segment. When the integrity value of the data segment is affirmatively verified, the method continues with the DS processing module generating a new set of encoded data slices. The method continues with the DS processing module verifying concurrency of the set of encoded data slices with the new set of encoded data slices and for each encoded data slice having a negative concurrency verification, flagging the encoded data slice as being potentially compromised.

Claims

1. A method for execution by a processing module, the method comprises:

selecting a data segment for determining whether at least a portion of the data segment has been compromised;

verifying integrity values of at least some encoded data slices of a set of encoded data slices, wherein the data segment is encoded in accordance with a dispersed storage error encoding function to produce the set of encoded data slices, which is stored in distributed storage (DS) units of a distributed storage network (DSN); and

when the integrity value of each of a decode threshold number of encoded data slices of the at least some of the set of encoded data slices are affirmatively verified;

verifying an integrity value of the data segment;

when the integrity value of the data segment is affirmatively verified, generating a new set of encoded data slices for the data segment in accordance with the dispersed storage error encoding function;

verifying concurrency of encoded data slices of the set of encoded data slices with corresponding encoded data slices of the new set of encoded data slices; and

for each of the encoded data slices having a negative concurrency verification, flagging the each of the encoded data slices as being potentially compromised.

2. The method of claim 1, wherein the verifying integrity values of at least some encoded data slices comprises:

selecting a first combination of encoded data slices of the set of encoded data slices totaling the decode threshold number of encoded data slices;

when the integrity value of each encoded data slice of the first combination of encoded data slices are affirmatively verified, establishing the that integrity values of at least some encoded data slices are affirmatively verified; and

when the integrity value of an encoded data slice of the first combination of encoded data slices is negatively verified to produce a potentially compromised encoded data slice, entering a loop that includes;

selecting another combination of encoded data slices of the set of encoded data slices totaling the decode threshold number of encoded data slices as a current combination of encoded data slices, wherein the another combination of encoded data slices excludes the potentially compromised encoded data slice;

when the integrity value of each encoded data slice of the current combination of encoded data slices are affirmatively verified, establishing the that integrity values of at least some encoded data slices are affirmatively verified and exiting the loop; and

when the integrity value of an encoded data slice of the current combination of encoded data slices is negatively verified to produce another potentially compromised encoded data slice, repeating the loop excluding the other potentially compromised encoded data slice.

3. The method of claim 1, wherein the verifying the integrity value of the data segment comprises:

dispersed storage error decoding the decode threshold number of encoded data slices to produce a recaptured data segment;

calculating an integrity value for the recaptured data segment;

comparing the calculated integrity value with a retrieved integrity value of the data segment; and

when the comparison is favorable, establishing that the integrity value of the data segment is verified.

4. The method of claim 1, wherein the verifying the integrity value of the data segment comprises:

retrieving a data segment integrity value from each of the decode threshold number of encoded data slices; and

when the data segment integrity value from each of the decode threshold number of encoded data slices substantially match, establishing that the integrity value of the data segment is verified.

5. The method of claim 1, wherein the verifying concurrency of encoded data slices of the set of encoded data slices with corresponding encoded data slices of the new set of encoded data slices comprises at least one of:

performing a bit by bit comparison of each encoded data slice of the set of encoded data slices with the corresponding encoded data slices of the new set of encoded data slices to verify the concurrency; and

comparing integrity values of each encoded data slice of the set of encoded data slices with integrity values of the corresponding encoded data slices of the new set of encoded data slices to verify the concurrency.

6. The method of claim 1 further comprises:

for each encoded data slice of the at least some encoded data slices that fails the verifying of the integrity value, flagging the failed encoded data slice as being potentially compromised.

7. The method of claim 1 further comprises:

when the integrity value of the data segment is negatively verified, entering a loop that includes;

selecting another combination of encoded data slices of the set of encoded data slices totaling the decode threshold number of encoded data slices as a current combination of encoded data slices;

verifying another integrity value of the data segment based on the other combination of encoded data slices;

when the other integrity value of the data segment based on the other combination of encoded data slices is affirmatively verified, establishing that the other integrity value of the data segment is affirmatively verified and exiting the loop; and

when the other integrity value of the data segment based on the other combination of encoded data slices is negatively verified, repeating the loop.

8. The method of claim 1 further comprises:

when each of the encoded data slices of the set of encoded data slices does not include the integrity value or the data segment does not include the integrity value of the data segment;

generating a plurality of recaptured data segments from various combinations of encoded data slices of the set of encoded data slices;

generating a plurality of data segment integrity values for the plurality of recaptured data segments;

when the plurality of data segment integrity values substantially match, indicating the set of encoded data slices has not been compromised; and

when a data segment integrity value of the plurality of data segment integrity values does not substantially match to other data segment integrity values of the plurality of data segment integrity values, identifying, based on the data segment integrity value, an encoded data slice of the set of encoded data slices as being potentially compromised.

9. The method of claim 1, wherein the flagging the each of the encoded data slices as being potentially compromised further comprises at least one of:

flagging each of the potentially compromised encoded data slices for rebuilding;

deleting each of the potentially compromised encoded data slices and replacing each of the potentially compromised encoded data slices with corresponding encoded data slices of the new set of encoded data slices; and

deleting the set of encoded data slices and replacing the set of encoded data slices with the new set of encoded data slices.

10. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

select a data segment for determining whether at least a portion of the data segment has been compromised;

a second module, when operable within the computing device, causes the computing device to;

verify integrity values of at least some encoded data slices of a set of encoded data slices, wherein the data segment is encoded in accordance with a dispersed storage error encoding function to produce the set of encoded data slices, which is stored in distributed storage (DS) units of a distributed storage network (DSN); and

a third module, when operable within the computing device, causes the computing device to;

when the integrity value of each of a decode threshold number of encoded data slices of the at least some of the set of encoded data slices are affirmatively verified;

verify an integrity value of the data segment;

when the integrity value of the data segment is affirmatively verified, generate a new set of encoded data slices for the data segment in accordance with the dispersed storage error encoding function;

verify concurrency of encoded data slices of the set of encoded data slices with corresponding encoded data slices of the new set of encoded data slices; and

for each of the encoded data slices having a negative concurrency verification, flag the each of the encoded data slices as being potentially compromised.

11. The DS module of claim 10, wherein the second module functions to verify integrity values of at least some encoded data slices by:

selecting a first combination of encoded data slices of the set of encoded data slices totaling the decode threshold number of encoded data slices;

when the integrity value of each encoded data slice of the first combination of encoded data slices are affirmatively verified, establishing the that integrity values of at least some encoded data slices are affirmatively verified; and

when the integrity value of an encoded data slice of the first combination of encoded data slices is negatively verified to produce a potentially compromised encoded data slice, entering a loop that includes;

selecting another combination of encoded data slices of the set of encoded data slices totaling the decode threshold number of encoded data slices as a current combination of encoded data slices, wherein the another combination of encoded data slices excludes the potentially compromised encoded data slice;

when the integrity value of each encoded data slice of the current combination of encoded data slices are affirmatively verified, establishing the that integrity values of at least some encoded data slices are affirmatively verified and exiting the loop; and

when the integrity value of an encoded data slice of the current combination of encoded data slices is negatively verified to produce another potentially compromised encoded data slice, repeating the loop excluding the other potentially compromised encoded data slice.

12. The DS module of claim 10, wherein the third module functions to verify the integrity value of the data segment by:

dispersed storage error decoding the decode threshold number of encoded data slices to produce a recaptured data segment;

calculating an integrity value for the recaptured data segment;

comparing the calculated integrity value with a retrieved integrity value of the data segment; and

when the comparison is favorable, establishing that the integrity value of the data segment is verified.

13. The DS module of claim 10, wherein the third module functions to verify the integrity value of the data segment by:

retrieving a data segment integrity value from each of the decode threshold number of encoded data slices; and

when the data segment integrity value from each of the decode threshold number of encoded data slices substantially match, establishing that the integrity value of the data segment is verified.

14. The DS module of claim 10, wherein the third module functions to verify concurrency of encoded data slices of the set of encoded data slices with corresponding encoded data slices of the new set of encoded data slices by at least one of:

performing a bit by bit comparison of each encoded data slice of the set of encoded data slices with the corresponding encoded data slices of the new set of encoded data slices to verify the concurrency; and

comparing integrity values of each encoded data slice of the set of encoded data slices with integrity values of the corresponding encoded data slices of the new set of encoded data slices to verify the concurrency.

15. The DS module of claim 10 further comprises:

the second module further functions to, for each encoded data slice of the at least some encoded data slices that fails the verifying of the integrity value, flag the failed encoded data slice as being potentially compromised.

16. The DS module of claim 10 further comprises:

the third module further functions to, when the integrity value of the data segment is negatively verified, enter a loop where the third module further functions to;

select another combination of encoded data slices of the set of encoded data slices totaling the decode threshold number of encoded data slices as a current combination of encoded data slices;

verify another integrity value of the data segment based on the other combination of encoded data slices;

when the other integrity value of the data segment based on the other combination of encoded data slices is affirmatively verified, establish that the other integrity value of the data segment is affirmatively verified and exit the loop; and

when the other integrity value of the data segment based on the other combination of encoded data slices is negatively verified, repeat the loop.

17. The DS module of claim 10 further comprises:

the third module further functions to, when each of the encoded data slices of the set of encoded data slices does not include the integrity value or the data segment does not include the integrity value of the data segment;

generate a plurality of recaptured data segments from various combinations of encoded data slices of the set of encoded data slices;

generate a plurality of data segment integrity values for the plurality of recaptured data segments;

when the plurality of data segment integrity values substantially match, indicate the set of encoded data slices has not been compromised; and

when a data segment integrity value of the plurality of data segment integrity values does not substantially match to other data segment integrity values of the plurality of data segment integrity values, identify, based on the data segment integrity value, an encoded data slice of the set of encoded data slices as being potentially compromised.

18. The DS module of claim 10, wherein the third module functions to flag the each of the encoded data slices as being potentially compromised further by at least one of:

flagging each of the potentially compromised encoded data slices for rebuilding;

deleting each of the potentially compromised encoded data slices and replacing each of the potentially compromised encoded data slices with corresponding encoded data slices of the new set of encoded data slices; and

deleting the set of encoded data slices and replacing the set of encoded data slices with the new set of encoded data slices.