ACCELERATING TRANSFER PROTOCOLS

• US 20160044100A1
• Filed: 08/06/2014
• Published: 02/11/2016
• Est. Priority Date: 08/06/2014
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Tripathy, Tarun K., Smith, Brian R., Dinkar, Abhijit S.
• Granted Patent: US 9,917,894 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Techniques and mechanisms described herein facilitate the acceleration of data transfer protocols via client side de-duplication techniques for transmitting data from a client device to a networked storage system while maintaining protocol usage characteristics. while preserving the same presentation semantics to the users. A data chunk may be determined at a client device by parsing a data stream generated at the client device via a network storage protocol. A fingerprint for the data chunk may be identified by applying a hash function to the data chunk. A determination may be made as to whether the chunk is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network. The client device may transmit a block map update request message including information for updating a block map to the networked storage system.

Claims

1. A method comprising:

at a client device comprising a processor and memory, identifying a fingerprint for a designated data chunk by applying a hash function to the designated data chunk via a processor, the designated data chunk determined by parsing a data stream at the client device to identify a plurality of data chunks, the data stream generated at the client device via a network storage protocol selected from the group consisting of;

the Network File System (NFS) protocol and the Common Internet File System (CIFS) protocol;

determining whether the designated data chunk is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network; and

transmitting to the networked storage system a block map update request message including information for updating a block map at the networked storage system.

2. The method recited in claim 1, wherein the block map uniquely identifies the plurality of data chunks in order.

3. The method recited in claim 1, wherein the block map includes location information identifying a respective location of each data chunk in either non-volatile memory or secondary storage at the networked storage system.

4. The method recited in claim 1, the method further comprising:

when it is determined that the designated data chunk is not stored at the networked storage system, transmitting the designated chunk to the networked storage system for storage.

5. The method recited in claim 1, the method further comprising:

when it is determined that the designated data chunk is stored at the networked storage system, identifying an existing memory location associated with the designated data chunk, the existing memory location comprising a location in non-volatile memory or secondary storage at the networked storage system.

6. The method recited in claim 1, wherein determining whether the designated data chunk is stored at the networked storage system further comprises receiving a fingerprint status message from the networked storage system.

7. The method recited in claim 6, wherein the fingerprint status message indicates whether the designated data chunk associated with the fingerprint is stored at the networked storage system.

8. The method recited in claim 1, wherein the designated data chunk is determined via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams.

9. The method recited in claim 8, wherein the rolling hash parsing technique employs a Rabin-Karp algorithm.

10. The method recited in claim 1, wherein the networked storage system is operable to store deduplicated data based on storage requests received via the network.

11. The method recited in claim 1, wherein the hash function is selected from a group consisting of:

a Secure Hash Algorithm (SHA) based function and a Message Digest (MD) based function.

12. A device comprising:

memory configured to store a fingerprint for a designated data chunk by applying a hash function to the designated data chunk via a processor, the designated data chunk determined by parsing a data stream at the device to identify a plurality of data chunks, the data stream generated at the device via a network storage protocol selected from the group consisting of;

the Network File System (NFS) protocol and the Common Internet File System (CIFS) protocol;

a processor operable to determine whether the designated data chunk is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network; and

a communications interface operable to transmit to the networked storage system a block map update request message including information for updating a block map at the networked storage system.

13. The device recited in claim 12, wherein the block map uniquely identifies the plurality of data chunks in order.

14. The device recited in claim 12, wherein the block map includes location information identifying a respective location of each data chunk in either non-volatile memory or secondary storage at the networked storage system.

15. The device recited in claim 12, wherein the processor is further operable to:

transmit the designated chunk to the networked storage system for storage when it is determined that the designated data chunk is not stored at the networked storage system.

16. The device recited in claim 12, wherein the processor is further operable to:

identify an existing memory location associated with the designated data chunk when it is determined that the designated data chunk is stored at the networked storage system, the existing memory location comprising a location in non-volatile memory or secondary storage at the networked storage system.

17. The device recited in claim 12, wherein determining whether the designated data chunk is stored at the networked storage system further comprises receiving a fingerprint status message from the networked storage system.

18. The device recited in claim 17, wherein the fingerprint status message indicates whether the designated data chunk associated with the fingerprint is stored at the networked storage system.

19. The device recited in claim 12, wherein the designated data chunk is determined via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams.

20. One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

at a client device comprising a processor and memory, identifying a fingerprint for a designated data chunk by applying a hash function to the designated data chunk via a processor, the designated data chunk determined by parsing a data stream at the client device to identify a plurality of data chunks, the data stream generated at the client device via a network storage protocol selected from the group consisting of;

the Network File System (NFS) protocol and the Common Internet File System (CIFS) protocol;

determining whether the designated data chunk is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network; and

transmitting to the networked storage system a block map update request message including information for updating a block map at the networked storage system.