FIXED SIZE EXTENTS FOR VARIABLE SIZE DEDUPLICATION SEGMENTS

• US 20130238570A1
• Filed: 03/08/2012
• Published: 09/12/2013
• Est. Priority Date: 03/08/2012
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Rao, Goutham, Jayaraman, Vinod
• Granted Patent: US 9,256,609 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Mechanisms are provided for maintaining variable size deduplication segments using fixed size extents. Variable size segments are identified and maintained in a datastore suitcase. Duplicate segments need not be maintained redundantly but can be managed by updating reference counts associated with the segments in the datastore suitcase. Segments are maintained using fixed size extents. A minor increase in storage overhead removes the need for inefficient recompaction when a segment is removed from the datastore suitcase. Fixed size extents can be reallocated for storage of new segments.

Claims

1. A method comprising:

receiving a data stream at an interface;

performing segment boundary identification and segment fingerprinting to generate a plurality of variable size segments including a first segment;

accessing a deduplication dictionary to determine whether the first segment is maintained in one or more datastore suitcases;

determining a first number of fixed sized extents associated with the one or more datastore suitcases needed to maintain the first segment.

2. The method of claim 1, wherein the plurality of variable size segments includes a second segment having a second segment size different from a first segment size.

3. The method of claim 1, wherein a first reference count is associated with the first segment.

4. The method of claim 3, wherein the first segment is deleted when the first reference count indicates that the segment is no longer mapped to any file.

5. The method of claim 1, wherein the first segment is maintained in a plurality of contiguous fixed size extents in a single datastore suitcase.

6. The method of claim 5., wherein after the first segment is deleted, the plurality of contiguous fixed size extents in the datastore suitcase used to maintain the now deleted first segment are used to maintain portions of a plurality of new segments without having to perform recompaction.

7. The method of claim 1, wherein the first segment is maintained in a plurality of non-contiguous fixed size extents in one or more datastore suitcases.

8. The method of claim 7, wherein after the first segment is deleted, the plurality of non-contiguous fixed size extents in the one or more datastore suitcases used to maintain the now deleted first segment are used to maintain portions of a plurality of new segments without having to perform recompaction.

9. The method of claim 1, wherein the deduplication dictionary is accessed using a first fingerprint corresponding to the first segment.

10. A system comprising:

an interface operable to receive a data stream;

a processor operable to perform segment boundary identification and segment fingerprinting to generate a plurality of variable size segments including a first segment and access a deduplication dictionary to determine whether the first segment is maintained in one or more datastore suitcases;

wherein a first number of fixed sized extents associated with the one or more datastore suitcases needed to maintain the first segment is determined.

11. The system of claim 10, wherein the plurality of variable size segments includes a second segment having a second segment size different from a first segment size.

12. The system of claim 10, wherein a first reference count is associated with the first segment.

13. The system of claim 12, wherein the first segment is deleted when the first reference count indicates that the segment is no longer mapped to any file.

14. The system of claim 10, wherein the first segment is maintained in a plurality of contiguous fixed size extents in a single datastore suitcase.

15. The system of claim 14, wherein after the first segment is deleted, the plurality of contiguous fixed size extents in the datastore suitcase used to maintain the now deleted first segment are used to maintain portions of a plurality of new segments without having to perform recompaction.

16. The system of claim 10, wherein the first segment is maintained in a plurality of non-contiguous fixed size extents in one or more datastore suitcases.

17. The system of claim 16, wherein after the first segment is deleted, the plurality of non-contiguous fixed size extents in the one or more datastore suitcases used to maintain the now deleted first segment are used to maintain portions of a plurality of new segments without having to perform recompaction.

18. The system of claim 10, wherein the deduplication dictionary is accessed using a first fingerprint corresponding to the first segment.

19. A computer readable storage medium comprising:

computer code for receiving a data stream at an interface;

computer code for performing segment boundary identification and segment fingerprinting to generate a plurality of variable size segments including a first segment;

computer code for accessing a deduplication dictionary to determine whether the first segment is maintained in one or more datastore suitcases;

computer code for determining a first number of fixed sized extents associated with the one or more datastore suitcases needed to maintain the first segment.

20. The computer readable medium of claim 19, wherein the plurality of variable size segments includes a second segment having a second segment size different from a first segment size.