TRANSFERRING DIFFERENCES BETWEEN CHUNKS DURING REPLICATION

• US 20150032978A1
• Filed: 07/26/2013
• Published: 01/29/2015
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Bashyam, Murali, Garigala, Sreekanth
• Granted Patent: US 10,146,787 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1752 based on file chunks

Abstract

Techniques and mechanisms described herein facilitate the replication of data between storage nodes. According to various embodiments, a request to provide a data chunk to a target storage node may be received at a source data storage node. A reference data chunk may be identified based on fingerprint information associated with the requested data chunk. The reference data chunk may be stored on the target storage node. The reference data chunk and the requested data chunk may each include a first data portion. Data chunk reconstruction information may be transmitted from the source data storage node to the target data storage node. The data chunk reconstruction information may identify the reference data chunk. The data chunk reconstruction information may include data difference information for constructing the requested data chunk at the target data storage node based on the reference data chunk.

Claims

1. A method comprising:

receiving, at a source data storage node, a request to provide a data chunk to a target storage node;

identifying a reference data chunk based on fingerprint information associated with the requested data chunk, the reference data chunk being stored on the target storage node, the reference data chunk and the requested data chunk each including a first data portion; and

transmitting data chunk reconstruction information from the source data storage node to the target data storage node, the data chunk reconstruction information identifying the reference data chunk, the data chunk reconstruction information including data difference information for constructing the requested data chunk at the target data storage node based on the reference data chunk.

2. The method recited in claim 1, wherein the data difference information includes a second data portion, the requested data chunk including the second data portion, the reference data chunk not including the second data portion.

3. The method recited in claim 1, wherein the requested chunk includes a plurality of subchunks, the first data portion including designated ones of the plurality of subchunks.

4. The method recited in claim 3, wherein the fingerprint information includes a hash value generated by applying a hash function to at least a portion of the requested data chunk.

5. The method recited in claim 4, wherein the requested chunk includes a plurality of subchunks, the first data portion including designated ones of the plurality of subchunks, the fingerprint information including a respective hash value for each of the plurality of subchunks, each respective hash value being generated by applying a hash function to the respective subchunk.

6. The method recited in claim 1, wherein identifying the reference data chunk comprises locating the fingerprint information in an index, the index mapping fingerprint information to data chunk portions.

7. The method recited in claim 6, wherein the identifying the reference data chunk further comprises identifying a plurality of related data chunks, the related data chunks including the reference data chunk, each of the plurality of related data chunks being associated with the fingerprint information in the index, each of the plurality of related data chunks having a respective data portion that overlaps with the requested data chunk.

8. The method recited in claim 1, wherein the reference data chunk and the requested data chunk are each stored in a storage system at the source data storage node, the storage system being operable to perform one or more operations for data deduplication.

9. The method recited in claim 8, wherein the target storage node is operable to replicate data stored on the source data storage node.

10. A system comprising:

a storage system operable to store a plurality of data chunks;

a processor operable to identify a reference data chunk based on fingerprint information associated with a data chunk requested by a target storage node, the reference data chunk being stored on the target storage node, the reference data chunk and the requested data chunk each including a first data portion; and

a communications interface operable to transmit data chunk reconstruction information to the target data storage node, the data chunk reconstruction information identifying the reference data chunk, the data chunk reconstruction information including data difference information for constructing the requested data chunk at the target data storage node based on the reference data chunk.

11. The system recited in claim 10, wherein the data difference information includes a second data portion, the requested data chunk including the second data portion, the reference data chunk not including the second data portion.

12. The system recited in claim 10, wherein the requested chunk includes a plurality of subchunks, the first data portion including designated ones of the plurality of subchunks.

13. The system recited in claim 12, wherein the fingerprint information includes a hash value generated by applying a hash function to at least a portion of the requested data chunk.

14. The system recited in claim 13, wherein the requested chunk includes a plurality of subchunks, the first data portion including designated ones of the plurality of subchunks, the fingerprint information including a respective hash value for each of the plurality of subchunks, each respective hash value being generated by applying a hash function to the respective subchunk.

15. The system recited in claim 10, wherein identifying the reference data chunk comprises locating the fingerprint information in an index, the index mapping fingerprint information to data chunk portions.

16. The system recited in claim 15, wherein the identifying the reference data chunk further comprises identifying a plurality of related data chunks, the related data chunks including the reference data chunk, each of the plurality of related data chunks being associated with the fingerprint information in the index, each of the plurality of related data chunks having a respective data portion that overlaps with the requested data chunk.

17. The system recited in claim 10, wherein the reference data chunk and the requested data chunk are each stored in a storage system, the storage system being operable to perform one or more operations for data deduplication.

18. The system recited in claim 17, wherein the target storage node is operable to replicate data stored on the source data storage node.

19. One or more computer readable media having instructions stored thereon for performing a method, the method comprising:

receiving, at a source data storage node, a request to provide a data chunk to a target storage node;

identifying a reference data chunk based on fingerprint information associated with the requested data chunk, the reference data chunk being stored on the target storage node, the reference data chunk and the requested data chunk each including a first data portion; and

transmitting data chunk reconstruction information from the source data storage node to the target data storage node, the data chunk reconstruction information identifying the reference data chunk, the data chunk reconstruction information including data difference information for constructing the requested data chunk at the target data storage node based on the reference data chunk.

20. The one or more computer readable media recited in claim 19, wherein the data difference information includes a second data portion, the requested data chunk including the second data portion, the reference data chunk not including the second data portion.