ENCRYPTION AND COMPRESSION OF DATA FOR STORAGE

• US 20090190760A1
• Filed: 01/28/2008
• Published: 07/30/2009
• Est. Priority Date: 01/28/2008
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Bojinov, Hristo, Wood, Robert Paul, Frandzel, Yuval
• Granted Patent: US 8,300,823 B2
• Time in Patent Office: Days
• US Class Current: 380/269
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0676 Magnetic disk device
• CPC Class Codes: H04L 2209/12 Details relating to cryptographic hardware or logic circuitry
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 9/065 Encryption by serially and continuously modifying data stream elements, e.g. stream cipher systems, RC4, SEAL or A5/3
• CPC Class Codes: H04L 9/3247 involving digital signatures

Abstract

Described is a process for encrypting and compressing data for storage by computing systems. A data stream received by a computing system is separated into data chunks for storage on a storage device using a data layout format. Each data chunk is examined and if possible, compressed to produce a compressed data chunk. Whether or not the data chunk can be compressed, the data chunk is encrypted and stored on the storage device in addition to metadata that describes the data chunk. When reading the stored data chunk from the storage device, the metadata identifies the starting location of the stored data chunk and provides metadata information to the computing system for decryption and decompression processing.

Claims

1. A computer-implemented method of writing a data stream, comprising:

receiving at least one data stream, wherein the at least one data stream can be partitioned into a plurality of data chunks;

compressing each of the plurality of data chunks;

encrypting each of the plurality of data chunks; and

storing each of the plurality of data chunks with metadata on a storage device using a particular data layout format, wherein the metadata describes attributes of a data chunk.

2. The computer-implemented method of claim 1, wherein the metadata includes compression information.

3. The computer-implemented method of claim 2, wherein the compression information identifies a compression process.

4. The computer-implemented method of claim 2, wherein the compression information further includes the maximum size of a written data chunk.

5. The computer-implemented method of claim 2, wherein the metadata includes file type information.

6. The computer-implemented method of claim 5, wherein the file type information includes an identifier for one of audio-video file, a still-image file and an audio file.

7. The computer-implemented method of claim 2, wherein the metadata includes authentication information.

8. The computer-implemented method of claim 7, wherein the authentication information includes a signature.

9. The computer-implemented method of claim 1, wherein each of the plurality of data chunks can be stored as uncompressed data chunks if a data chunk cannot be compressed.

10. The computing-implemented method of claim 1, wherein a particular data layout format is identified by a computing system in response to receiving a data access request.

11. The computer-implemented method of claim 10, wherein one of a plurality of data layout formats includes at least one metadata and data chunk pair having gaps between a plurality of metadata and data chunk pairs.

12. The computer-implemented method of claim 10, wherein one of a plurality of data layout formats includes at least one metadata and data chunk pair having no gaps between a plurality of metadata and data chunk pairs.

13. The computer-implemented method of claim 10, wherein one of a plurality of data layout formats includes metadata indexing a data chunk.

14. A system for writing a data stream, comprising:

a compression module, wherein the compression module compresses a data chunk of the data stream;

an encryption module, wherein the encryption module encrypts the data chunk of the data stream compressed by the compression module; and

storing the data chunk with metadata on a storage device on a storage device using a particular data layout format, wherein the metadata describes attributes of the data chunk.

15. The system for writing a data stream of claim 14, wherein the compression module initiates a compression algorithm based on a file type associated with the data stream.

16. The system for writing a data stream of claim 15, wherein the file type includes one of a video file type, an audio file type, an image file type, and a document file type.

17. The system of claim 14, wherein the particular data layout format is identified in response to receiving a data access request.

18. The system of claim 17, wherein one of a plurality of data layout formats includes at least one metadata and data chunk pair having gaps between a plurality of metadata and data chunk pairs.

19. The system of claim 17, wherein one of a plurality of data layout formats includes at least one metadata and data chunk pair having no gaps between a plurality of metadata and data chunk pairs.

20. The system of claim 17, wherein one of a plurality of data layout formats includes metadata indexing a data chunk.

21. The system for writing a data stream of claim 14, wherein the metadata includes compression information.

22. The system for writing a data stream of claim 21, wherein the compression information further includes the maximum size of a written data chunk.

23. The system for writing a data stream of claim 22, wherein the metadata includes file type information.

24. A computing system for storing data, comprising:

a processor for compressing a data chunk of a data stream, wherein a compressed data chunk is encrypted for storage; and

a storage device of the computing system, wherein an encrypted data chunk and metadata are stored on the storage device using a particular data layout format.

25. The computing system for storing data of claim 24, wherein a particular data layout format is identified by the computing system in response to receiving a data access request.

26. The computing system for storing data of claim 25, wherein one of a plurality of data layout formats includes at least one metadata and data chunk pair having gaps between a plurality of metadata and data chunk pairs.

27. The computing system for storing data of claim 25, wherein one of a plurality of data layout formats includes at least one metadata and data chunk pair having no gaps between a plurality of metadata and data chunk pairs.

28. The computing system for storing data of claim 24, wherein the metadata includes compression information.

29. The computing system for storing data of claim 24, wherein the metadata includes file type information.

30. The computing system for storing data of claim 24, wherein the metadata is stored in an index file to identify the location of a stored data chunk on the storage device.

31. The computer-implemented method of claim 24, wherein the data chunk can be stored as an uncompressed data chunk if the data chunk cannot be compressed.

32. A machine-implemented method, comprising:

determining the beginning of a stored data chunk stored on a storage device by examining metadata describing the stored data chunk, wherein the metadata is stored on the storage device according to a data layout format;

reading the stored data chunk from the storage device;

decrypting the stored data chunk to produce a compressed data chunk; and

decompressing the compressed data chunk using the metadata.