Adaptive replication for segmentation anchoring type

• US 10,331,362 B1
• Filed: 09/30/2016
• Issued: 06/25/2019
• Est. Priority Date: 09/30/2016
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Parekh, Harshadrai
• Primary Examiner(s): Birkhimer, Christopher D
• Application Number:
  US15/282,967
• Time in Patent Office: 998 Days
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Described is a system for identifying data that may differ between files used as part of a replication process. The system may determine a type of segmentation used for segmenting data such as a variable size segmentation or a fixed size segmentation is used. Based on the segmentation of a file, the system may identify segments that may be modified. For example, the system may identify only the particular modified segments within a boundary when a fixed size segmentation is used identify all of the segments within a boundary as modified when a variable segmentation is used. Accordingly, depending on the scenario, the system may determine an efficient mechanism for identifying data to send to a target storage as part of a replication process.

Claims

1. A computer-implemented method, comprising:

segmenting a file to be stored on a source storage system into segments;

comparing a first segment tree of the file with a second segment tree of the file, the second segment tree representing a basefile of the file, wherein the first and second segment trees each include at least a first level representing the segments of the file, and a second level representing fingerprints of the segments within one or more fixed size boundaries;

determining the file has been modified based on the comparison of fingerprints for the first and second segment trees corresponding to a first fixed size boundary of the one or more fixed size boundaries;

identifying segments of the file within the first fixed size boundary of the first segment tree as modified based on whether the segments of the file within the first fixed size boundary and corresponding segments of the basefile are in alignment; and

sending the segments identified as modified within the first fixed size boundary from the source storage system to a target storage system for replication.

2. The method of claim 1, wherein segmenting the file to be stored on a source storage system includes segmenting the file into fixed size segments or variable size segments.

3. The method of claim 2, wherein identifying the segments of the file as modified includes,

determining whether the file is segmented into fixed size segments or variable size segments,

if the file is segmented into fixed size segments, selecting only the modified segments within the first fixed size boundary as the segments identified as modified based on the segments of the file and the basefile being in alignment within the first boundary, or

if the file is segmented into variable size segments, selecting all of the segments within the first fixed size boundary as the segments identified as modified based on the segments of the file and the basefile not being in alignment within the boundary.

4. The method of claim 3, wherein determining whether the file is segmented into fixed size segments or variable size segments is based on an attribute associated with the file that indicates whether the file is segmented into fixed size or variable size segments.

5. The method of claim 1, wherein sending the segments identified as modified from the source storage system to a target storage system for replication is performed in response to determining the target storage system includes a version of the basefile.

6. The method of claim 2, wherein the first boundary has a first predetermined size, and each of the fixed size segments has a second predetermined size.

7. The method of claim 6, wherein each of the variable size segments has a size in a predetermined range.

8. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising:

segmenting a file to be stored on a source storage system into segments;

comparing a first segment tree of the file with a second segment tree of the file, the second segment tree representing a basefile of the file, wherein the first and second segment trees each include at least a first level representing the segments of the file, and a second level representing fingerprints of the segments within one or more fixed size boundaries;

determining the file has been modified based on the comparison of fingerprints for the first and second segment trees corresponding to a first fixed size boundary of the one or more fixed size boundaries;

identifying segments of the file within the first fixed size boundary of the first segment tree as modified based on whether the segments of the file within the first fixed size boundary and corresponding segments of the basefile are in alignment; and

sending the segments identified as modified within the first fixed size boundary from the source storage system to a target storage system for replication.

9. The medium of claim 8, wherein segmenting the file to be stored on a source storage system includes segmenting the file into fixed size segments or variable size segments.

10. The medium of claim 9, wherein identifying the segments of the file as modified includes,

determining whether the file is segmented into fixed size segments or variable size segments,

if the file is segmented into fixed size segments, selecting only the modified segments within the first fixed size boundary as the segments identified as modified based on the segments of the file and the basefile being in alignment within the first boundary, or

if the file is segmented into variable size segments, selecting all of the segments within the first fixed size boundary as the segments identified as modified based on the segments of the file and the basefile not being in alignment within the boundary.

11. The medium of claim 10, wherein determining whether the file is segmented into fixed size segments or variable size segments is based on an attribute associated with the file that indicates whether the file is segmented into fixed size or variable size segments.

12. The medium of claim 8, wherein sending the segments identified as modified from the source storage system to a target storage system for replication is performed in response to determining the target storage system includes a version of the basefile.

13. The medium of claim 9, wherein the first boundary has a first predetermined size, and each of the fixed size segments has a second predetermined size.

14. The medium of claim 13, wherein each of the variable size segments has a size in a predetermined range.

15. A device, comprising:

a memory storing instructions; and

a processor coupled to the memory to execute the instructions from the memory, the processor being configured to perform operations, the operations comprising;

segmenting a file to be stored on a source storage system into segments;

comparing a first segment tree of the file with a second segment tree of the file the second segment tree representing a basefile of the file, wherein the first and second segment trees each include at least a first level representing the segments of the file, and a second level representing fingerprints of the segments within one or more fixed size boundaries;

determining the file has been modified based on the comparison of fingerprints for the first and second segment trees corresponding to a first fixed size boundary of the one or more fixed size boundaries;

identifying segments of the file within the first fixed size boundary of the first segment tree as modified based on whether the segments of the file within the first fixed size boundary and corresponding segments of the basefile are in alignment; and

sending the segments identified as modified within the first fixed size boundary from the source storage system to a target storage system for replication.

16. The device of claim 15, wherein segmenting the file to be stored on a source storage system includes segmenting the file into fixed size segments or variable size segments.

17. The device of claim 16, wherein identifying the segments of the file as modified includes,

determining whether the file is segmented into fixed size segments or variable size segments,

if the file is segmented into fixed size segments, selecting only the modified segments within the first fixed size boundary as the segments identified as modified based on the segments of the file and the basefile being in alignment within the first boundary, or

if the file is segmented into variable size segments, selecting all of the segments within the first fixed size boundary as the segments identified as modified based on the segments of the file and the basefile not being in alignment within the boundary.

18. The device of claim 17, wherein determining whether the file is segmented into fixed size segments or variable size segments is based on an attribute associated with the file that indicates whether the file is segmented into fixed size or variable size segments.

19. The device of claim 15, wherein sending the segments identified as modified from the source storage system to a target storage system for replication is performed in response to determining the target storage system includes a version of the basefile.

20. The device of claim 16, wherein the first boundary has a first predetermined size, and each of the fixed size segments has a second predetermined size, and wherein each of the variable size segments has a size in a predetermined range.