Accelerating transfer protocols
First Claim
1. A method executed by a client device comprising a processor and memory, the method comprising:
- identifying a fingerprint for a designated data chunk by applying a hash function to the designated data chunk, the designated data chunk being determined by parsing a data stream at the client device to identify a plurality of data chunks, wherein the designated data chunk is determined via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams, the data stream generated at the client device via a first network storage protocol;
determining whether a data chunk associated with the fingerprint is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network, wherein data chunks are stored at the networked storage system according to a second network storage protocol, wherein the networked storage system parses data streams into data chunks via the rolling hash parsing technique used by the client device and the networked storage system determines fingerprints for data chunks using the hash function used by the client device;
responsive to a determination that a data chunk associated with the fingerprint is stored at the networked storage system, transmitting to a virtual file system of the networked storage system, a block map update information for updating a block map at the networked storage system, wherein the block map update information includes a first block map entry that points to the data chunk associated with the fingerprint that is stored at the networked storage system; and
when it is determined that a data chunk associated with the fingerprint is not stored at the networked storage system, transmitting the designated data chunk to the networked storage system for storage, along with a second block map update information including a second block map entry that points to the storage location of the transmitted designated data chunk.
23 Assignments
0 Petitions
Accused Products
Abstract
Techniques and mechanisms described herein facilitate the acceleration of data transfer protocols via client side de-duplication techniques for transmitting data from a client device to a networked storage system while maintaining protocol usage characteristics. while preserving the same presentation semantics to the users. A data chunk may be determined at a client device by parsing a data stream generated at the client device via a network storage protocol. A fingerprint for the data chunk may be identified by applying a hash function to the data chunk. A determination may be made as to whether the chunk is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network. The client device may transmit a block map update request message including information for updating a block map to the networked storage system.
-
Citations
16 Claims
-
1. A method executed by a client device comprising a processor and memory, the method comprising:
-
identifying a fingerprint for a designated data chunk by applying a hash function to the designated data chunk, the designated data chunk being determined by parsing a data stream at the client device to identify a plurality of data chunks, wherein the designated data chunk is determined via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams, the data stream generated at the client device via a first network storage protocol; determining whether a data chunk associated with the fingerprint is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network, wherein data chunks are stored at the networked storage system according to a second network storage protocol, wherein the networked storage system parses data streams into data chunks via the rolling hash parsing technique used by the client device and the networked storage system determines fingerprints for data chunks using the hash function used by the client device; responsive to a determination that a data chunk associated with the fingerprint is stored at the networked storage system, transmitting to a virtual file system of the networked storage system, a block map update information for updating a block map at the networked storage system, wherein the block map update information includes a first block map entry that points to the data chunk associated with the fingerprint that is stored at the networked storage system; and when it is determined that a data chunk associated with the fingerprint is not stored at the networked storage system, transmitting the designated data chunk to the networked storage system for storage, along with a second block map update information including a second block map entry that points to the storage location of the transmitted designated data chunk. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A device comprising:
-
memory configured to store a fingerprint for a designated data chunk, the designated data chunk determined by parsing a data stream at the device to identify a plurality of data chunks, the fingerprint determined by applying a hash function to the designated data chunk, wherein the designated data chunk is determined via a designated rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams, the data stream generated at the device via a first network storage; a processor operable to determine whether a data chunk associated with the fingerprint is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network, wherein data chunks are stored at the networked storage system according to a second network storage protocol, wherein the networked storage system parses data streams into data chunks via the designated rolling hash parsing technique used by the client device and the networked storage system determines fingerprints for data chunks using the hash function used by the client device; and a communications interface operable to; transmit to a virtual file system of the networked storage system, responsive to a determination that a data chunk associated with the fingerprint is stored at the networked storage system, a block map update information for updating a block map at the networked storage system, wherein the block map update information includes a first block map entry that points to the data chunk associated with the fingerprint that is stored at the networked storage system, and when it is determined that a data chunk associated with the fingerprint is not stored at the networked storage system, transmit the designated chunk to the networked storage system for storage, along with a second block map update information including a second block map entry that points to the storage location of the transmitted designated data chunk. - View Dependent Claims (11, 12, 13, 14, 15)
-
-
16. One or more non-transitory computer readable media having instructions stored thereon to cause a client device comprising a processor and memory to perform operations comprising:
-
identify a fingerprint for a designated data chunk by applying a hash function to the designated data chunk via the processor, the designated data chunk determined by parsing a data stream at the client device to identify a plurality of data chunks, wherein the designated data chunk is determined via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams, the data stream generated at the client device via a first network storage protocol; determine whether a data chunk associated with the fingerprint is stored at a networked storage system by transmitting the fingerprint to the networked storage system via a network, wherein data chunks are stored at the networked storage system according to a second network storage protocol, wherein the networked storage system is configured to parse data streams into data chunks via the rolling hash parsing technique used by the client device and the networked storage system determines fingerprints for data chunks using the hash function used by the client; responsive to a determination that the designated data chunk is stored at the networked storage system, transmit to a virtual file system of the networked storage system, a block map update information for updating a block map at the networked storage system, wherein the block map update information includes a first block map entry that points to the data chunk associated with the fingerprint that is stored at the networked storage system; and when it is determined that a data chunk associated with the fingerprint is not stored at the networked storage system, transmitting the designated chunk to the networked storage system for storage, along with a second block map update information including a second block map entry that points to the storage location of the transmitted designated data chunk.
-
Specification