Efficient storage of encrypted data in a dispersed storage network

• US 9,747,457 B2
• Filed: 05/06/2015
• Issued: 08/29/2017
• Est. Priority Date: 11/25/2009
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Grube, Gary W., Markison, Timothy W., Gladwin, S. Christopher, Abhijeet, Kumar, Dhuse, Greg, Resch, Jason K.
• Primary Examiner(s): Eskandarnia, Arvin
• Application Number:
  US14/705,752
• Publication Number:
  US 20150235032A1
• Time in Patent Office: 846 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/6272 by registering files or documents with a third party
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2221/2107 File encryption
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H04L 9/0861 Generation of secret information including derivation or calculation of cryptographic keys or passwords

Abstract

A method for storing a data object includes identifying data segments of the data object. The method continues with generating key indexes for the data segments. For a data segment, the method continues with accessing data segment key information based on a corresponding key index of the plurality of key indexes to determine whether an encryption key has been generated for a similar data segment. When the encryption key has been generated for the similar data segment, the method continues with using the encryption key to encrypt the data segment to produce an encrypted data segment. The method continues with compressing the encrypted data segment to produce a compressed and encrypted data segment. The method continues with storing the compressed and encrypted data segment in a storage unit of a dispersed storage network (DSN).

Claims

1. A method for storing a data object, the method comprises:

identifying a plurality of data segments of the data object;

generating a plurality of key indexes for the plurality of data segments;

for a data segment of the plurality of data segments;

accessing data segment key information based on a corresponding key index of the plurality of key indexes to determine whether an encryption key has been generated for a similar data segment, wherein the encryption key is a representation of the similar data segment;

when the encryption key has been generated for the similar data segment;

using the encryption key to encrypt the data segment to produce an encrypted data segment;

compressing the encrypted data segment to produce a compressed and encrypted data segment; and

storing the compressed and encrypted data segment in a storage unit of a dispersed storage network (DSN);

when the encryption key has not been generated for the similar data segment;

generating an encryption key based on a representation of the data segment;

generating the corresponding key index based on a representation of the encryption key;

updating the data segment key information to include the corresponding key index;

dispersed storage error encoding the encryption key to produce a set of encoded key slices;

storing the set of encoded key slices in a plurality of storage units of the DSN;

encrypting the data segment using the encryption key to produce the encrypted data segment;

compressing the encrypted data segment to produce the compressed and encrypted data segment; and

storing the compressed and encrypted data segment in the storage unit.

2. The method of claim 1, wherein generating the plurality of key indexes comprises:

generating the corresponding key index of the plurality of key indexes for the data segment by;

generating an encryption key based on a representation of the data segment; and

generating the corresponding key index based on a representation of the encryption key.

3. The method of claim 2 further comprises:

generating the encryption key by one of;

setting the encryption key substantially equal to the data segment;

and performing a mathematical function on the data segment to produce the encryption key;

generating the corresponding key index by one of;

setting the corresponding key index substantially equal to the encryption key;

performing a mathematical function on the corresponding key index to produce the encryption key; and

performing a hash function on the corresponding key index to produce the encryption key.

4. The method of claim 1, wherein the data segment key information comprises:

30 one or more data segment key tables.

5. A computer device comprises:

an interface;

memory; and

a processing module operably coupled to the interface and the memory, wherein the processing module is;

identifying a plurality of data segments of a data object;

generating a plurality of key indexes for the plurality of data segments;

for a data segment of the plurality of data segments;

accessing data segment key information based on a corresponding key index of the plurality of key indexes to determine whether an encryption key has been generated for a similar data segment,

wherein the encryption key is are presentation of the similar data segment;

when the encryption key has been generated for the similar data segment;

use the encryption key to encrypt the data segment to produce an encrypted data segment;

compress the encrypted data segment to produce a compressed and encrypted data segment; and

outputting, via the interface, the compressed and encrypted data segment to a storage unit of a dispersed storage network (DSN) for storage therein;

when the encryption key has not been generated for the similar data segment;

generating an encryption key based on a representation of the data segment;

generating the corresponding key index based on a representation of the encryption key;

update the data segment key information to include the corresponding key index;

dispersed storage error encode the encryption key to produce a set of encoded key slices;

outputting, via the interface, the set of encoded key slices in a plurality of storage units of the DSN for storage therein;

encrypting the data segment using the encryption key to produce the encrypted data segment;

compressing the encrypted data segment to produce the compressed and encrypted data segment; and

outputting, via the interface, the compressed and encrypted data segment to the storage unit for storage therein.

6. The computing device of claim 5, wherein the processing module further

generates the plurality of key indexes by:

generating the corresponding key index of the plurality of key indexes for the data segment by;

generating an encryption key based on a representation of the data segment; and

generating the corresponding key index based on a representation of the encryption key.

7. The computing device of claim 6, wherein the processing module is further:

generating the encryption key by one of;

setting the encryption key substantially equal to the data segment; and

performing a mathematical function on the data segment to produce the encryption key;

generating the corresponding key index by one of;

setting the corresponding key index substantially equal to the encryption key;

performing a mathematical function on the corresponding key index to produce the encryption key; and

performing a hash function on the corresponding key index to produce the encryption key.

8. The computing device of claim 5, wherein the data segment key information comprises:

one or more data segment key tables.

9. A non-transitory computer readable storage device comprises:

a first memory section that stores operational instructions that, when executed by a computing device, causes the computing device to;

identify a plurality of data segments of a data object;

a second memory section that stores operational instructions that, when executed by the computing device, causes the computing device to;

generate a plurality of key indexes for the plurality of data segments;

a third memory section that stores operational instructions that, when executed by the computing device, causes the computing device to;

for a data segment of the plurality of data segments;

 

access data segment key information based on a corresponding key index of the plurality of key indexes to determine whether an encryption key has been generated for a similar data segment, wherein the encryption key is a representation of the similar data segment;

 

when the encryption key has been generated for the similar data segment;

 

use the encryption key to encrypt the data segment to produce an encrypted data segment;

 

compress the encrypted data segment to produce a compressed and encrypted data segment; and

 

output, via an interface, the compressed and encrypted data segment to a storage unit of a dispersed storage network (DSN) for storage therein;

wherein the third memory section further stores operational instructions that, when executed by the computing device, causes the computing device to;

when the encryption key has not been generated for the similar data segment;

generate an encryption key based on a representation of the data segment;

generate the corresponding key index based on a representation of the encryption key;

update the data segment key information to include the corresponding key index;

dispersed storage error encode the encryption key to produce a set of encoded key slices;

output, via the interface, the set of encoded key slices in a plurality of storage units of the DSN for storage therein;

encrypt the data segment using the encryption key to produce the encrypted data segment;

compress the encrypted data segment to produce the compressed and encrypted data segment; and

output, via the interface, the compressed and encrypted data segment to the storage unit for storage therein.

10. The non-transitory computer readable storage device of claim 9, wherein the second memory section further stores operational instructions that, when executed by the computing device, causes the computing device to generate the plurality of key indexes by:

generating the corresponding key index of the plurality of key indexes for the data segment by;

generating an encryption key based on a representation of the data segment; and

generating the corresponding key index based on a representation of the encryption key.

11. The non-transitory computer readable storage device of claim 9, wherein the second memory section further stores operational instructions that, when executed by the computing device, causes the computing device to:

generate the encryption key by one of;

setting the encryption key substantially equal to the data segment; and

performing a mathematical function on the data segment to produce the encryption key;

generate the corresponding key index by one of;

setting the corresponding key index substantially equal to the encryption key;

performing a mathematical function on the corresponding key index to produce the encryption key; and

performing a hash function on the corresponding key index to produce the encryption key.

12. The non-transitory computer readable storage device of claim 9, wherein the data segment key information comprises:

one or more data segment key tables.