Securing encoding data slices using an integrity check value list

• US 9,933,969 B2
• Filed: 11/21/2016
• Issued: 04/03/2018
• Est. Priority Date: 11/30/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: McShane, Niall J., Resch, Jason K., Volvovski, Ilya
• Primary Examiner(s): Tu, Christine T.
• Application Number:
  US15/357,293
• Publication Number:
  US 20170153948A1
• Time in Patent Office: 498 Days
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H03M 13/616 Matrix operations, especially for generator matrices or check matrices, e.g. column or row permutations
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561

Abstract

A method includes retrieving a read threshold number of integrity check value list (ICVL) encoded data slices of a set of ICVL encoded data slices. The method further includes determining whether an appended ICVL of each ICVL encoded data slice of the read threshold number of ICVL encoded data slices substantially match. When the appended ICVL of one of the ICVL encoded does not substantially match the appended ICVL of other ICVL encoded data slices, the method further includes determining a likely cause for the mismatch. When the likely cause is missing a revision update, the method further includes initiate rebuilding of the encoded data slice portion. The method further includes generating an integrity check value for the rebuilt encoded data slice and updating the integrity check value list. The method further includes appending the updated integrity check value list to the rebuilt encoded data slice.

Claims

1. A method comprises:

retrieving, by a computing device of a dispersed storage network (DSN) or another computing device of the DSN, a read threshold number of integrity check value list (ICVL) encoded data slices of a set of ICVL encoded data slices from at least some storage units of a set of storage units;

determining, by the computing device or the other computing device, whether an appended ICVL of each ICVL encoded data slice of the read threshold number of ICVL encoded data slices substantially match;

when the appended ICVL of one of the ICVL encoded data slices of the read threshold number of ICVL encoded data slices does not substantially match the appended ICVL of other ICVL encoded data slices of the read threshold number of ICVL encoded data slices;

determining, by the computing device or the other computing device, a likely cause for the appended ICVL of the one of the ICVL encoded data slices not substantially matching the appended ICVL of the other ICVL encoded data slices; and

when the likely cause is missing a revision update;

initiating, by the computing device or the other computing device, rebuilding of the encoded data slice portion of the one of the ICVL encoded data slices to produce a rebuilt encoded data slice;

generating, by the computing device or the other computing device, an integrity check value for the rebuilt encoded data slice;

generating, by the computing device or the other computing device, an updated integrity check value list including the integrity check value for the rebuilt encoded data slice and integrity check values of the encoded data slices of the other ICVL encoded data slices;

appending, by the computing device or the other computing device, the updated integrity check value list to the rebuilt encoded data slice to produce a rebuild ICVL encoded data slice; and

sending, by the computing device or the other computing device, the rebuilt ICVL encoded data slice to one of the storage units of the set of storage units for storage therein.

2. The method of claim 1 further comprises:

generating, by the computing device, a set of integrity check values for a set of encoded data slices, wherein a data segment of a data object is dispersed storage error encoded to produce the set of encoded data slices;

generating, by the computing device, an integrity check value list from the set of integrity check values;

appending, by the computing device, the integrity check value list (ICVL) to each encoded data slice of the set of encoded data slices to produce a set of ICVL encoded data slices; and

sending, by the computing device, the set of ICVL encoded data slices to the set of storage units of the DSN for storage therein.

3. The method of claim 1 further comprises:

when the likely cause is an inaccurate integrity check value list (ICVL) appended to the one of the ICVL encoded data slices;

calculating a new integrity check value for an encoded data slice portion of the one of the ICVL encoded data slices;

extracting the integrity check value from the appended ICVL of one of the other ICVL encoded data slices to produce an extracted integrity check value;

comparing the new integrity check value with the extracted integrity check value; and

when the new integrity check value substantially matches the extracted integrity check value, utilizing the encoded data slice in decoding of the read threshold number of encoded data slices to recover the data segment.

4. The method of claim 3 further comprises:

when the new integrity check value does not substantially match the extracted integrity check value;

initiating, by the computing device or the other computing device, rebuilding of the encoded data slice portion of the one of the ICVL encoded data slices to produce a rebuilt encoded data slice;

generating, by the computing device or the other computing device, an integrity check value for the rebuilt encoded data slice;

generating, by the computing device or the other computing device, an updated integrity check value list including the integrity check value for the rebuilt encoded data slice and integrity check values of the encoded data slices of the other ICVL encoded data slices;

appending, by the computing device or the other computing device, the updated integrity check value list to the rebuilt encoded data slice to produce a rebuild ICVL encoded data slice; and

sending, by the computing device or the other computing device, the rebuilt ICVL encoded data slice one of the storage units of the set of storage units for storage therein.

5. The method of claim 1 further comprises:

when the likely cause is an unauthorized modification of the one of the ICVL encoded data slices, sending, by the computing device or the other computing device, a delete message to a storage unit storing the one of the ICVL encoded data slices to delete the one of the ICVL encoded data slices.

6. The method of claim 1 further comprises:

when the likely cause is an unauthorized modification of the one of the ICVL encoded data slices;

initiating, by the computing device or the other computing device, rebuilding of the encoded data slice portion of the one of the ICVL encoded data slices to produce a rebuilt encoded data slice;

generating, by the computing device or the other computing device, an integrity check value for the rebuilt encoded data slice;

generating, by the computing device or the other computing device, an updated integrity check value list including the integrity check value for the rebuilt encoded data slice and integrity check values of the encoded data slices of the other ICVL encoded data slices;

appending, by the computing device or the other computing device, the updated integrity check value list to the rebuilt encoded data slice to produce a rebuild ICVL encoded data slice; and

sending, by the computing device or the other computing device, the rebuilt ICVL encoded data slice one of the storage units of the set of storage units for storage therein.

7. The method of claim 1 further comprises:

when the likely cause is an unauthorized modification of the one of the ICVL encoded data slices, determining, by the computing device or the other computing device, whether other encoded data slices stored by a storage unit storing the one of the ICVL encoded data slices have been modified in an unauthorized manner; and

when the other encoded data slices stored by the storage unit have been modified in the unauthorized manner, initiating, by the computing device or the other computing device, at least one of a decommissioning of the storage unit, a quarantine of the storage unit, and a higher level of security regarding access to the storage unit.

8. A computer readable memory comprises:

a first memory element that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN) or another computing device of the DSN, causes the computing device or the other computing device to;

retrieve a read threshold number of integrity check value list (ICVL) encoded data slices of a set of ICVL encoded data slices from at least some storage units of a set of storage units;

determine whether an appended ICVL of each ICVL encoded data slice of the read threshold number of ICVL encoded data slices substantially match;

when the appended ICVL of one of the ICVL encoded data slices of the read threshold number of ICVL encoded data slices does not substantially match the appended ICVL of other ICVL encoded data slices of the read threshold number of ICVL encoded data slices, determine a likely cause for the appended ICVL of the one of the ICVL encoded data slices not substantially matching the appended ICVL of the other ICVL encoded data slices;

a second memory element that stores operational instructions that, when executed by the computing device or the computing device, causes the computing device or the other computing device to;

when the likely cause is missing a revision update;

initiate rebuilding of the encoded data slice portion of the one of the ICVL encoded data slices to produce a rebuilt encoded data slice;

generate an integrity check value for the rebuilt encoded data slice;

generate an updated integrity check value list including the integrity check value for the rebuilt encoded data slice and integrity check values of the encoded data slices of the other ICVL encoded data slices;

append the updated integrity check value list to the rebuilt encoded data slice to produce a rebuild ICVL encoded data slice; and

send the rebuilt ICVL encoded data slice to one of the storage units of the set of storage units for storage therein.

9. The computer readable memory of claim 8 further comprises:

a third memory element that stores operational instructions that, when executed by the computing device, causes the computing device to;

generate a set of integrity check values for a set of encoded data slices, wherein a data segment of a data object is dispersed storage error encoded to produce the set of encoded data slices;

generate an integrity check value list from the set of integrity check values;

append the integrity check value list (ICVL) to each encoded data slice of the set of encoded data slices to produce a set of ICVL encoded data slices; and

send the set of ICVL encoded data slices to the set of storage units of the DSN for storage therein.

10. The computer readable memory of claim 8 further comprises:

a third memory element that stores operational instructions that, when executed by the computing device or the computing device, causes the computing device or the other computing device to;

when the likely cause is an inaccurate integrity check value list (ICVL) appended to the one of the ICVL encoded data slices;

calculate a new integrity check value for an encoded data slice portion of the one of the ICVL encoded data slices;

extract the integrity check value from the appended ICVL of one of the other ICVL encoded data slices to produce an extracted integrity check value;

compare the new integrity check value with the extracted integrity check value; and

when the new integrity check value substantially matches the extracted integrity check value, utilize the encoded data slice in decoding of the read threshold number of encoded data slices to recover the data segment.

11. The computer readable memory of claim 10, wherein the third memory element further stores operational instructions that, when executed by the computing device or the computing device, causes the computing device or the other computing device to:

when the new integrity check value does not substantially match the extracted integrity check value;

initiate rebuilding of the encoded data slice portion of the one of the ICVL encoded data slices to produce a rebuilt encoded data slice;

generate an integrity check value for the rebuilt encoded data slice;

generate an updated integrity check value list including the integrity check value for the rebuilt encoded data slice and integrity check values of the encoded data slices of the other ICVL encoded data slices;

append the updated integrity check value list to the rebuilt encoded data slice to produce a rebuild ICVL encoded data slice; and

send the rebuilt ICVL encoded data slice one of the storage units of the set of storage units for storage therein.

12. The computer readable memory of claim 8 further comprises:

a third memory element that stores operational instructions that, when executed by the computing device or the computing device, causes the computing device or the other computing device to;

when the likely cause is an unauthorized modification of the one of the ICVL encoded data slices, send a delete message to a storage unit storing the one of the ICVL encoded data slices to delete the one of the ICVL encoded data slices.

13. The computer readable memory of claim 8 further comprises:

a third memory element that stores operational instructions that, when executed by the computing device or the computing device, causes the computing device or the other computing device to;

when the likely cause is an unauthorized modification of the one of the ICVL encoded data slices;

initiate rebuilding of the encoded data slice portion of the one of the ICVL encoded data slices to produce a rebuilt encoded data slice;

generate an integrity check value for the rebuilt encoded data slice;

generate an updated integrity check value list including the integrity check value for the rebuilt encoded data slice and integrity check values of the encoded data slices of the other ICVL encoded data slices;

append the updated integrity check value list to the rebuilt encoded data slice to produce a rebuild ICVL encoded data slice; and

send the rebuilt ICVL encoded data slice one of the storage units of the set of storage units for storage therein.

14. The computer readable memory of claim 8 further comprises:

a third memory element that stores operational instructions that, when executed by the computing device or the computing device, causes the computing device or the other computing device to;

when the likely cause is an unauthorized modification of the one of the ICVL encoded data slices, determine whether other encoded data slices stored by a storage unit storing the one of the ICVL encoded data slices have been modified in an unauthorized manner; and

when the other encoded data slices stored by the storage unit have been modified in the unauthorized manner, initiate at least one of a decommissioning of the storage unit, a quarantine of the storage unit, and a higher level of security regarding access to the storage unit.