Network storage server with integrated encryption, compression and deduplication capability

• US 9,395,929 B2
• Filed: 04/25/2008
• Issued: 07/19/2016
• Est. Priority Date: 04/25/2008
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Bojinov, Hristo, Subramanian, Ananthan
• Primary Examiner(s): Hoffman, Brandon
• Assistant Examiner(s): Salehi, Helai
• Application Number:
  US12/110,046
• Publication Number:
  US 20090268903A1
• Time in Patent Office: 3,007 Days
• Field of Search: 713/150, 713164-167
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/264 Remote server
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0625 Power saving in storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 2209/60 Digital content management, e.g. content distribution
• CPC Class Codes: H04L 9/0836 using tree structure or hierarchical structure
• CPC Class Codes: H04L 9/0894 Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A network storage server receives multiple write requests from a set of clients via a network and internally buffers multiple data blocks written by the write requests. At a consistency point, the storage server commits the data blocks to a nonvolatile mass storage facility. The consistency point process includes using a storage operating system in the network storage server to compress the data blocks, encrypt selected data blocks, and store the compressed and (possibly) encrypted data blocks in the nonvolatile mass storage facility. Data blocks can also be fingerprinted in parallel with compression and/or encryption, to facilitate subsequent deduplication. Data blocks can be indexed and classified according to content or attributes of the data. Encryption can be applied at different levels of logical container granularity, where a separate, unique cryptographic key is used for each encrypted data container.

Claims

1. A method of operation of a network storage server, the method comprising:

receiving a plurality of write requests at the network storage server from a set of clients;

buffering a plurality of data blocks written by the write requests in a cache in the network storage server;

during a consistency point process that is characterized by committing the plurality of buffered data blocks written by the write requests and stored in the cache to a nonvolatile mass storage facility, using a storage operating system in the network storage server to,

compress each of the data blocks,

encrypt each of the data blocks,

for each of the data blocks, in parallel with compressing and encrypting the data block, hash an unencrypted version of the data block to generate a fingerprint of the data block, and

store the compressed and encrypted data blocks in the nonvolatile mass storage facility; and

enabling at least one of the encrypted data blocks to be shared by a plurality of logical containers, including using the fingerprints of the data blocks to identify duplicate data blocks.

2. A method as recited in claim 1, wherein using the storage operating system to encrypt each of the data blocks comprises using a file system manager to encrypt the data blocks.

3. A method as recited in claim 1, wherein enabling at least one of the encrypted data blocks to be shared by a plurality of logical containers comprises:

storing the fingerprint of each data block in a memory; and

coalescing the duplicate data blocks.

4. A method as recited in claim 1, wherein the network storage server maintains data in a hierarchical structure that includes a plurality of direct blocks as a lowest level of the hierarchical structure and a plurality of levels of indirect blocks logically above the direct blocks in the hierarchical structure.

5. A method as recited in claim 1, wherein using the storage operating system comprises using a plurality of encryption keys to encrypt data blocks.

6. A method as recited in claim 1, wherein the storage operating system maintains a plurality of logical containers at a plurality of levels of granularity;

the method further comprising enabling logical containers at each of the plurality of levels of granularity to be encrypted, by enabling encryption keys to be assigned to logical containers in each of the plurality of levels of granularity.

7. A method as recited in claim 1, further comprising using the storage operating system in the network storage server to digitally sign each of the data blocks, prior to storing each of the compressed and encrypted data blocks in the nonvolatile mass storage facility.

8. A method as recited in claim 1, further comprising:

responding to a read request from one of the set of clients, by using the storage operating system in the network storage server to,

access one of the compressed and encrypted data blocks in the nonvolatile mass storage facility;

decrypt and decompress the accessed data block; and

send the decrypted and decompressed data block to the client.

9. A method as recited in claim 2, wherein using the storage operating system to compress each of the data blocks comprises using the file system manager to compress the data blocks.

10. A method as recited in claim 3, further comprising:

generating a digital signature for each said fingerprint, prior to storing the fingerprint in the memory; and

using the digital signatures of the fingerprints to detect tampering in one of the fingerprints.

11. A method as recited in claim 4, wherein using the storage operating system to encrypt each of the data blocks comprises encrypting direct data blocks affected by the write requests but not indirect blocks affected by the write requests.

12. A method as recited in claim 4, wherein using the storage operating system to encrypt each of the data blocks comprises encrypting direct data blocks affected by the write requests and indirect blocks affected by the write requests.

13. A method as recited in claim 4, further comprising determining how many levels of indirect data blocks to encrypt, based upon a user-specified setting.

14. A method as recited in claim 5, wherein using the storage operating system to encrypt each of the data blocks comprises, for each of the data blocks, using a virtual volume block number of the data block and an encryption key to encrypt the data block, the virtual volume block number representing an address of the data block within a virtual volume that contains the data block.

15. A method as recited in claim 5,

wherein the storage operating system maintains data in a plurality of volumes, each volume containing a plurality of data blocks;

and

wherein using the storage operating system to encrypt each of the data blocks comprises using a different encryption key for each of the volumes.

16. A method as recited in claim 6, for each logical container to which an encryption key is assigned, storing said encryption key in a metadata container associated with the logical container.

17. A method as recited in claim 5, further comprising encrypting each of the encryption keys with a wrapper encryption key.

18. A method as recited in claim 5, further comprising changing an encryption key to be used for a set of data blocks at each of a plurality of consistency points.

19. A method as recited in claim 6, wherein enabling logical containers at each of the plurality of levels of granularity to be encrypted comprises:

encrypting data at two or more levels of granularity in accordance with user inputs that specify logical containers to be encrypted at two or more levels of granularity.

20. A method as recited in claim 19, wherein the plurality of levels of granularity comprise file level, directory level and volume level.

21. A method as recited in claim 20, wherein enabling logical containers at each of the plurality of levels of granularity to be encrypted comprises:

encrypting data blocks of a file with a first encryption key; and

encrypting a directory which contains the file with a second encryption key.

22. A method as recited in claim 21, wherein enabling logical containers at each of the plurality of levels of granularity to be encrypted further comprises:

encrypting a volume which contains the directory with a third encryption key.

23. A non-transitory machine-readable medium comprising program code, the program code to:

receive a plurality of write requests at a network storage server from a set of clients;

buffer a plurality of data blocks written by the write requests in a cache in the network storage server;

during a consistency point process that is characterized by committing the plurality of buffered data blocks written by the write requests and stored in the cache to a nonvolatile mass storage facility, use a storage operating system in the network storage server to,

compress each of the data blocks,

encrypt each of the data blocks,

for each of the data blocks, in parallel with compression and encryption of the data block, hash an unencrypted version of the data block to generate a fingerprint of the data block, and

store the compressed and encrypted data blocks in the nonvolatile mass storage facility; and

enable at least one of the encrypted data blocks to be shared by a plurality of logical containers, including using the fingerprints of the data blocks to identify duplicate data blocks.

24. An apparatus comprising:

a processor; and

a machine-readable medium having program code executable by the processor to cause the apparatus to;

receive a plurality of write requests at the apparatus from a set of clients;

buffer a plurality of data blocks written by the write requests in a cache in the apparatus;

during a consistency point process that is characterized by committing the plurality of buffered data blocks written by the write requests and stored in the cache to a nonvolatile mass storage facility, use a storage operating system in the apparatus to,

compress each of the data blocks,

encrypt each of the data blocks,

for each of the data blocks, in parallel with compression and encryption of the data block, hash an unencrypted version of the data block to generate a fingerprint of the data block, and

store the compressed and encrypted data blocks in the nonvolatile mass storage facility; and

enable at least one of the encrypted data blocks to be shared by a plurality of logical containers, including using the fingerprints of the data blocks to identify duplicate data blocks.