SEGMENT DEDUPLICATION SYSTEM WITH COMPRESSION OF SEGMENTS

• US 20140330797A1
• Filed: 05/06/2014
• Published: 11/06/2014
• Est. Priority Date: 06/09/2009
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Lumb, Christopher R.
• Granted Patent: US 9,436,723 B2
• Time in Patent Office: Days
• US Class Current: 707/693
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3064 Segmenting

Abstract

A system for storing compressed data comprises a processor and a memory. The processor is configured to receive a compressed segment. The compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments. The processor is further configured to determine whether the compressed segment has been previously stored, and in the event that the compressed segment has not been previously stored, store the compressed segment. The memory is coupled to the processor and configured to provide the processor with instructions.

Claims

1. A system for storing compressed data, comprising:

a processor configured to;

receive a compressed segment, wherein the compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments;

determine whether the compressed segment has been previously stored;

in the event that the compressed segment has not been previously stored, store the compressed segment; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. A system as in claim 1, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

3. A system as in claim 1, wherein compressing comprises compressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

4. A system as in claim 1, wherein for a first segment and a second segment that is identical to the first segment, a compressed first segment is identical to a compressed second segment.

5. A method for storing compressed data, comprising:

receiving a compressed segment, wherein the compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments;

determining, using a processor, whether the compressed segment has been previously stored;

in the event that the compressed segment has not been previously stored, storing the compressed segment.

6. A method as in claim 5, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

7. A method as in claim 5, wherein compressing comprises compressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

8. A method as in claim 5, wherein for a first segment and a second segment that is identical to the first segment, a compressed first segment is identical to a compressed second segment.

9. A computer program product for storing compressed data, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

receiving a compressed segment, wherein the compressed segment is determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments;

determining, using a processor, whether the compressed segment has been previously stored;

in the event that the compressed segment has not been previously stored, storing the compressed segment.

10. A system for reading compressed data, comprising:

a deduplicated storage device;

a processor configured to;

retrieve one or more compressed segments from the deduplicated storage device, wherein the one or more compressed segments were determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments;

decompress the one or more compressed segments; and

assemble the one or more compressed segments to reconstruct the data stream, the data block, or the data file; and

a memory coupled to the processor and configured to provide the processor with instructions.

11. A system as in claim 10, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

12. A system as in claim 10, wherein decompressing comprises decompressing using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

13. A system as in claim 10, wherein for a first compressed segment and a second compressed segment that is identical to the first compressed segment, a decompressed first compressed segment is identical to a decompressed second compressed segment.

14. A method for reading compressed data, comprising:

retrieving one or more compressed segments from a deduplicated storage device, wherein the one or more compressed segments were determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments;

decompressing, using a processor, the one or more compressed segments; and

assembling the one or more decompressed segments to reconstruct the data stream, the data block, or the data file.

15. A method as in claim 14, wherein the breaking the data stream, the data block, or the data file is based at least in part on one of the following:

a content-based technique or a non-content-based technique.

16. A method as in claim 14, wherein decompressing comprises decrypting using one of the following:

a lossless compression system, a Huffman coding system, or a Lempel-Ziv Welch coding system.

17. A method as in claim 14, wherein for a first compressed segment and a second compressed segment that is identical to the first compressed segment, a decompressed first compressed segment is identical to a decompressed second compressed segment.

18. A computer program product for reading compressed data, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

retrieving one or more compressed segments from a deduplicated storage device, wherein the one or more compressed segments were determined by breaking a data stream, a data block, or a data file into one or more segments and compressing each of the one or more segments;

decompressing, using a processor, the one or more compressed segments; and

assembling the one or more decompressed segments to reconstruct the data stream, the data block, or the data file.