Segment deduplication system with encryption and compression of segments

• US 20100313040A1
• Filed: 06/09/2009
• Published: 12/09/2010
• Est. Priority Date: 06/09/2009
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Data Domain, Inc. (Dell Technologies Inc.)
• Inventors: Lumb, Christopher R.
• Granted Patent: US 8,731,190 B2
• Time in Patent Office: Days
• US Class Current: 713/193
• CPC Class Codes: G06F 11/1402 Saving, restoring, recovering or retrying
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 2209/34 Encoding or coding, e.g. Huffman coding or error correction
• CPC Class Codes: H04L 9/0618 Block ciphers, i.e. encrypting groups of characters of a plain text message using fixed encryption transformation

Abstract

A system for storing encrypted compressed data comprises a processor and a memory. The processor is configured to determine whether an encrypted compressed segment has been previously stored. The encrypted compressed segment was determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments. The processor is further configured to store the encrypted compressed segment in the event that the encrypted compressed segment has not been previously stored. The memory is coupled to the processor and configured to provide the processor with instructions.

Claims

1. A system for storing encrypted compressed data, comprising:

a processor configured to;

determine whether an encrypted compressed segment has been previously stored, wherein the encrypted compressed segment was determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments;

in the event that the encrypted compressed segment has not been previously stored, store the encrypted compressed segment; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. A system as in claim 1, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

3. A system as in claim 1, wherein compressing comprises compressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

4. A system as in claim 1, wherein encrypting comprises encrypting using one of the following:

a stream cipher system, a cipher feedback system, an electronic code book system, an AES system, or a DES system.

5. A system as in claim 1, wherein for a first segment and a second segment that is identical to the first segment, an encrypted compressed first segment is identical to an encrypted compressed second segment.

6. A method for storing encrypted compressed data,.comprising:

determining, using a processor, whether an encrypted compressed segment has been previously stored, wherein the encrypted compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments;

in the event that the encrypted compressed segment has not been previously stored, storing the encrypted compressed segment.

7. A method as in claim 6, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

8. A method as in claim 6, wherein compressing comprises compressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

9. A method as in claim 6, wherein encrypting comprises encrypting using one of the following:

a stream cipher system, a cipher feedback system, an electronic code book system, an AES system, or a DES system.

10. A method as in claim 6, wherein for a first segment and a second segment that is identical to the first segment, an encrypted compressed first segment is identical to an encrypted compressed second segment.

11. A computer program product for storing encrypted compressed data, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

determining, using a processor, whether an encrypted compressed segment has been previously stored, wherein the encrypted compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments;

in the event that the encrypted compressed segment has not been previously stored, storing the encrypted compressed segment.

12. A system for reading encrypted compressed data, comprising:

a deduplicated storage device;

a processor configured to;

decrypt one or more encrypted compressed segments stored in the deduplicated storage device, wherein the one or more encrypted compressed segments were determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments;

decompressing the one or more decrypted compressed segments; and

assemble the one or more decrypted decompressed segments to reconstruct the data stream, the data block, or the data file; and

a memory coupled to the processor and configured to provide the processor with instructions.

13. A system as in claim 12, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

14. A system as in claim 12, wherein compressing comprises compressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

15. A system as in claim 12, wherein decrypting comprises decrypting using one of the following:

a stream cipher system, a cipher feedback system, an electronic code book system, an AES system, or a DES system.

16. A system as in claim 12, wherein for a first encrypted compressed segment and a second encrypted compressed segment that is identical to the first encrypted compressed segment, a decrypted decompressed first encrypted compressed segment is identical to a decrypted decompressed second encrypted compressed segment.

17. A method for reading encrypted data, comprising:

decrypting, using a processor, one or more encrypted compressed segments stored in a deduplicated storage device, wherein the one or more encrypted compressed segments were determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments;

decompressing the one or more decrypted compressed segments; and

assembling the one or more decrypted decompressed segments to reconstruct the data stream, the data block, or the data file.

18. A method as in claim 17, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

19. A method as in claim 17, wherein compressing comprises compressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

20. A method as in claim 17, wherein decrypting comprises decrypting using one of the following:

a stream cipher system, a cipher feedback system, an electronic code book system, an AES system, or a DES system.

21. A method as in claim 17, wherein for a first encrypted compressed segment and a second encrypted compressed segment that is identical to the first encrypted compressed segment, a decrypted decompressed first encrypted compressed segment is identical to a decrypted decompressed second encrypted decompressed segment.

22. A computer program product for reading encrypted compressed data, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

decrypting, using a processor, one or more encrypted compressed segments stored on a deduplicated storage device, wherein the one or more encrypted compressed segments were determined by breaking a data stream, a data block, or a data file into one or more segments and compressing and then encrypting each of the one or more segments;

decompressing the one or more decrypted compressed segments; and

assembling the one or more decrypted decompressed segments to reconstruct the data stream, the data block, or the data file.