Deduplication of encrypted dataset on datadomain backup appliance

• US 9,225,691 B1
• Filed: 09/27/2013
• Issued: 12/29/2015
• Est. Priority Date: 09/27/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Balasubramanian, Shankar, Duggal, Abhinav, Krishnappa, Bharath, Sharda, Ravi
• Primary Examiner(s): Siddiqi, Mohammad A
• Application Number:
  US14/040,407
• Time in Patent Office: 823 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: H04L 2209/603 Digital right managament [DRM]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H04L 9/3236 using cryptographic hash functions
• CPC Class Codes: H04L 9/3239 involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD

Abstract

Exemplary methods for deduplicating encrypted files are described herein. The exemplary methods include receiving a first encrypted data file from a remote source that is encrypted by a first security key. In one embodiment, the methods include transmitting to a remote security manager a first key identifier (ID) that is extracted from the first data file, the first key ID identifying the first security key. In one aspect of the invention, in response to receiving the first security key from the remote security manager based on the first key ID, decrypting the first data file using the first security key provided by the remote security manager. In at least one embodiment, the methods include deduplicating the decrypted first data file.

Claims

1. A computer-implemented method for deduplicating encrypted data, the method comprising:

receiving a first data file from a remote source to be stored at a local deduplicated storage system by a backup and restore (backup/restore) engine executed by a processor of the local deduplicated storage system, wherein the first data file is encrypted by a first security key;

transmitting by a security module of the backup/restore engine to a remote security manager a first key identifier (ID) that is extracted from the first data file, the first key ID identifying the first security key, wherein the remote security manager is hosted by a remote server separated from the remote source and the local deduplicated storage system;

in response to receiving the first security key from the remote security manager based on the first key ID, decrypting by the security module the first data file using the first security key provided by the remote security manager;

deduplicating the decrypted first data file by a deduplication storage engine executed by the processor of the local deduplicated storage system, wherein deduplicating the decrypted first data file comprises partitioning the decrypted first data file into a plurality of data segments;

generating a hash for each of plurality of data segments,

comparing the generated hashes with hashes of data segments already stored at the local deduplicated storage system, and

determining one or more deduplicated data segments of the first data file, wherein the deduplicated data segments of the first data file are data segments with hashes that do not match the hashes of data segments already stored at the local deduplicated storage system;

encrypting the deduplicated data segments of the first data file using a second security key; and

storing the encrypted deduplicated data segments of the first data file.

2. The method of claim 1, wherein the first and second security keys are different.

3. The method of claim 2, further comprising:

decrypting, in response to a request to restore the first data file, the stored deduplicated data segments of the first data file using the second security key.

4. The method of claim 3, further comprising:

generating the first data file using the decrypted deduplicated data segments of the first data file.

5. The method of claim 4, wherein generating the first data file comprises:

encrypting the generated first data file using the first security key.

6. The method of claim 1, wherein the first and second security keys are the same.

7. A non-transitory computer-readable medium having computer instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

receiving a first data file from a remote source to be stored at a local deduplicated storage system by a backup and restore (backup/restore) engine executed by a processor of the local deduplicated storage system, wherein the first data file is encrypted by a first security key;

transmitting by a security module of the backup/restore engine to a remote security manager a first key identifier (ID) that is extracted from the first data file, the first key ID identifying the first security key, wherein the remote security manager is hosted by a remote server separated from the remote source and the local deduplicated storage system;

in response to receiving the first security key from the remote security manager based on the first key ID, decrypting by the security module the first data file using the first security key provided by the remote security manager;

deduplicating the decrypted first data file by a deduplication storage engine executed by the processor of the local deduplicated storage system, wherein deduplicating the decrypted first data file comprises partitioning the decrypted first data file into a plurality of data segments;

generating a hash for each of plurality of data segments,

comparing the generated hashes with hashes of data segments already stored at the local deduplicated storage system, and

determining one or more deduplicated data segments of the first data file, wherein the deduplicated data segments of the first data file are data segments with hashes that do not match the hashes of data segments already stored at the local deduplicated storage system;

encrypting the deduplicated data segments of the first data file using a second security key; and

storing the encrypted deduplicated data segments of the first data file.

8. The non-transitory computer-readable medium of claim 7, wherein the first and second security keys are different.

9. The non-transitory computer-readable medium of claim 8, wherein the operations further comprise:

decrypting, in response to a request to restore the first data file, the stored deduplicated data segments of the first data file using the second security key.

10. The non-transitory computer-readable medium of claim 9, wherein the operations further comprise:

generating the first data file using the decrypted deduplicated data segments of the first data file.

11. The non-transitory computer-readable medium of claim 10, wherein generating the first data file comprises:

encrypting the generated first data file using the first security key.

12. The non-transitory computer-readable medium of claim 7, wherein the first and second security keys are the same.

13. A data processing system, comprising:

a processor;

a network interface configured to receive a first data file from a remote source to be stored at a local deduplicated storage system, wherein the first data file is encrypted by a first security key;

a security module executed by the processor and coupled to the network interface, configured to transmit to a remote security manager a first key identifier (ID) that is extracted from the first data file, the first key ID identifying the first security key, and in response to receiving the first security key from the remote security manager based on the first key ID, decrypt the first data file using the first security key provided by the remote security manager, wherein the remote security manager is hosted by a remote server separated from the remote source and the local deduplicated storage system; and

a deduplication storage engine executed by the processor and coupled to the security module, configured to deduplicate the decrypted first data file, including

partitioning the decrypted first data file into a plurality of data segments;

generating a hash for each of plurality of data segments,

comparing the generated hashes with hashes of data segments already stored at the local deduplicated storage system, and

determining one or more deduplicated data segments of the first data file, wherein the deduplicated data segments of the first data file are data segments with hashes that do not match the hashes of data segments already stored at the local deduplicated storage system,

wherein the security module encrypts the deduplicated data segments of the first data file using a second security key and stores the encrypted deduplicated data segments of the first data file.

14. The data processing system of claim 13, wherein the first and second security keys are different.

15. The data processing system of claim 14, wherein the security module is further configured to:

decrypt, in response to a request to restore the first data file, the stored deduplicated data segments of the first data file using the second security key.

16. The data processing system of claim 15, wherein the deduplication storage engine is further configured to:

generate the first data file using the decrypted deduplicated data segments of the first data file.

17. The data processing system of claim 16, wherein the security module is further configured to:

encrypt the generated first data file using the first security key.

18. The data processing system of claim 13, wherein the first and second security keys are the same.