CHUNK COMPRESSION IN A DEDUPLICATION AWARE CLIENT ENVIRONMENT

• US 20160041970A1
• Filed: 08/06/2014
• Published: 02/11/2016
• Est. Priority Date: 08/06/2014
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Tripathy, Tarun K., Smith, Brian R., Dinkar, Abhijit S.
• Granted Patent: US 9,990,352 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 40/205 Parsing

Abstract

Techniques and mechanisms described herein facilitate the transmission of a data stream to a networked storage system. According to various embodiments, a data stream may be parsed to identify one or more uncompressed data chunks for transmission to a networked storage system. Each uncompressed data chunk may be compressed to produce a respective compressed data chunk. Each compressed data chunk may be transmitted to the networked storage system via a network for storage at the networked storage system.

Claims

1. A method comprising:

at a client device comprising a processor and memory, parsing a data stream to identify one or more uncompressed data chunks for transmission to a networked storage system;

compressing each uncompressed data chunk to produce a respective compressed data chunk; and

transmitting each compressed data chunk to the networked storage system via a network for storage at the networked storage system.

2. The method recited in claim 1, the method further comprising:

identifying an amount of available computing resources at the client device.

3. The method recited in claim 2, the method further comprising:

identifying a computing resource availability compression threshold at the client device, wherein each uncompressed data chunk is compressed when it is determined that the amount of available computing resources meets or exceeds the computing resource availability compression threshold.

4. The method recited in claim 1, the method further comprising:

identifying an amount of available computing resources at the networked storage system.

5. The method recited in claim 1, wherein one or more uncompressed data chunks are identified via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams.

6. The method recited in claim 1, the method further comprising:

identifying a respective fingerprint for each uncompressed data chunk by applying a hash function to the uncompressed data chunk.

7. The method recited in claim 6, the method further comprising:

determining whether the uncompressed data chunk is stored at a networked storage system by transmitting each fingerprint to the networked storage system via the network.

8. The method recited in claim 7, the method further comprising:

transmitting a block map update request message to the networked storage system, the block map update request message including information for updating a block map at the networked storage system, the block map identifying a designated memory location at which each data compressed chunk is stored at the networked storage system.

9. The method recited in claim 7, wherein determining whether the uncompressed chunk is stored at the networked storage system further comprises receiving a fingerprint status message from the networked storage system, the fingerprint status message indicating whether the chunk associated with the fingerprint is stored at the networked storage system.

10. The method recited in claim 1, the method further comprising:

encrypting each compressed data chunk prior to transmission.

11. The method recited in claim 1, wherein the networked storage system is operable to store deduplicated data based on storage requests received via the network.

12. The method recited in claim 1, the data stream is generated at the client device via a network storage protocol selected from the group consisting of:

the Network File System (NFS) protocol, the Common Internet File System (CIFS) protocol, and the Open Storage (OST) protocol.

13. A device comprising:

memory operable to store one or more uncompressed data chunks for transmission to a networked storage system, the one or more uncompressed data chunks identified by parsing a data stream;

a processor operable to compress each uncompressed data chunk to produce a respective compressed data chunk; and

a communications interface operable to transmit each compressed data chunk to the networked storage system via a network for storage at the networked storage system.

14. The device recited in claim 13, wherein the processor is further operable to:

identify an amount of available computing resources at the client device.

15. The device recited in claim 14, wherein the processor is further operable to:

identify a computing resource availability compression threshold at the client device, wherein each uncompressed data chunk is compressed when it is determined that the amount of available computing resources meets or exceeds the computing resource availability compression threshold.

16. The device recited in claim 13, wherein the processor is further operable to:

identify an amount of available computing resources at the networked storage system.

17. The device recited in claim 13, wherein one or more uncompressed data chunks are identified via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams.

18. The device recited in claim 13, wherein the processor is further operable to:

identify a respective fingerprint for each uncompressed data chunk by applying a hash function to the uncompressed data chunk.

19. The device recited in claim 18, wherein the processor is further operable to:

determine whether the uncompressed data chunk is stored at a networked storage system by transmitting each fingerprint to the networked storage system via the network.

20. One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

at a client device comprising a processor and memory, parsing a data stream to identify one or more uncompressed data chunks for transmission to a networked storage system;

compressing each uncompressed data chunk to produce a respective compressed data chunk; and

transmitting each compressed data chunk to the networked storage system via a network for storage at the networked storage system.