Verifying data security in a dispersed storage network

• US 10,104,045 B2
• Filed: 04/18/2010
• Issued: 10/16/2018
• Est. Priority Date: 04/20/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K.
• Primary Examiner(s): Jacob, Ajith
• Application Number:
  US12/762,350
• Publication Number:
  US 20100268692A1
• Time in Patent Office: 3,103 Days
• Field of Search: 707687
• CPC Class Codes: G06F 11/1044 with specific ECC/EDC distribution
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2211/1059 Parity-single bit-RAID5, i.e. RAID 5 implementations
• CPC Class Codes: H04L 2209/043 of tables, e.g. lookup, substitution or mapping
• CPC Class Codes: H04L 2209/12 Details relating to cryptographic hardware or logic circuitry
• CPC Class Codes: H04L 2209/603 Digital right managament [DRM]
• CPC Class Codes: H04L 2209/608 Watermarking
• CPC Class Codes: H04L 2209/80 Wireless
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/085 Secret sharing or secret splitting, e.g. threshold schemes
• CPC Class Codes: H04L 9/3236 using cryptographic hash functions

Abstract

An integrity record is appended to data slices prior to being sent to multiple slice storage units. Each of the data slices includes a different encoded version of the same data segment. An integrity indicator of each data slice is computed, and the integrity record is generated based on each of the individual integrity indicators, and may be, for example, list or a hash of the combined integrity indicators. When retrieving data slices from storage, the integrity record can be stripped off, a new integrity indicator of the data slice calculated, and a new integrity record created. The new integrity record can be compared to the original integrity record, and used to verify the integrity of the data slices.

Claims

1. A method comprising:

encoding a data segment of data in accordance with a dispersed storage error encoding function to produce a plurality of data slices;

for each data slice of the plurality of data slices, calculating, using a hash function, an integrity indicator to a plurality of integrity indicators, wherein the integrity indicator is representative of an original, error-free, version of a corresponding data slice of the plurality of data slices;

performing a function on the plurality of integrity indicators to produce an integrity record, wherein the integrity record is representative of an original, error-free, version of the plurality of data slices;

creating a plurality of copies of the integrity record;

for each of the plurality of data slices, appending one of the plurality of copies of the integrity record to a corresponding one of the plurality of data slices to produce a plurality of modified data slices, wherein a modified data slice of the plurality of modified data slices includes the corresponding data slice, the integrity indicator that represents the original, error-free, version of the corresponding data slice, and the integrity record that represents the original, error-free, version of the plurality of data slices; and

transmitting the plurality of modified data slices to slice storage units for storage therein.

2. The method of claim 1 further comprising:

caching the plurality of integrity indicators; and

performing an aggregation function that aggregates the plurality of integrity indicators to produce the integrity record.

3. The method of claim 1, further comprising:

caching the plurality of integrity indicators; and

performing a hash function that hashes the plurality of integrity indicators to produce the integrity record.

4. The method of claim 1, wherein the calculating the plurality of integrity indicators further comprising:

performing a hash function on a data slice of the plurality of data slices to produce one of the plurality of integrity indicators, wherein the hash function includes one or more of double hashing, checksums, a Hashed Message Authentication Code (HMAC), and digital signatures.

5. An apparatus comprising:

a processor operable to;

encode a data segment of data in accordance with a dispersed storage error encoding function to produce a plurality of data slices;

for each data slice of the plurality of data slices, calculate, using a hash function, an integrity indicator to a plurality of integrity indicators, wherein the integrity indicator is representative of an original, error-free, version of a corresponding data slice of the plurality of data slices;

perform a function on the plurality of integrity indicators to produce an integrity record, wherein the integrity record is representative of an original, error-free, version of the plurality of data slices;

create a plurality of copies of the integrity record;

for each of the plurality of data slices, append one of the plurality of copies of the integrity record to a corresponding one of the plurality of data slices to produce a plurality of modified data slices, wherein a modified data slice of the plurality of modified data slices includes the corresponding data slice, the integrity indicator that represents the original, error-free, version of the corresponding data slice, and the integrity record that represents the original, error-free, version of the plurality of data slices; and

a communications interface operable to transmit the plurality of modified data slices to slice storage units for storage therein.

6. The apparatus of claim 5 further comprising:

cache memory to store the plurality of integrity indicators; and

the processor further to perform an aggregation function that aggregates the plurality of integrity indicators to produce the integrity record.

7. The apparatus of claim 5 further comprising:

cache memory to store the plurality of integrity indicators; and

the processor further to perform a hash function that hashes the plurality of integrity indicators to produce the integrity record.

8. The apparatus of claim 5, wherein the processor is further operable to calculate the plurality of integrity indicators by:

performing a hash function on a data slice of the plurality of data slices to produce one of the plurality of integrity indicators, wherein the hash function includes one or more of double hashing, checksums, a Hashed Message Authentication Code (HMAC), and digital signatures.