Managing deduplication density

• US 8,712,976 B1
• Filed: 03/28/2012
• Issued: 04/29/2014
• Est. Priority Date: 03/28/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Chen, Xiangping, Ruef, Richard P., Armangau, Philippe, Owen, Karl M., Ku, Mark K.
• Primary Examiner(s): Corrielus, Jean M
• Assistant Examiner(s): ASPINWALL, EVAN S
• Application Number:
  US13/432,312
• Time in Patent Office: 762 Days
• Field of Search: 707/692
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641

Abstract

A method is used in managing deduplication density. A deduplication density of a data object is determined. The deduplication density is indicated by a number of deduplicated data objects represented by the data object. Based on the deduplication density of the data object, the data object is skipped for applying a deduplicating technique to the data object. An association is created between the data object and another data object. The other data object is used for applying the deduplicating technique where contents of the other data object are identical to contents of the data object.

Claims

1. A method for use in managing deduplication density in a storage system, the method comprising:

determining a deduplication density of a first data object, wherein the deduplication density is indicated by a number of deduplicated data objects represented by the first data object;

based on the deduplication density of the first data object, skipping the first data object for applying a deduplicating technique to the first data object;

creating an association between the first data object and a second data object, wherein the second data object is used for applying the deduplicating technique, wherein data stored in the first data object is identical to data stored in the second data object, wherein creating the association between the first and second data objects include adding a first reference in the first data object pointing to the second data object and adding a second reference in the second data object pointing to the first data object; and

recovering a set of deduplicated data blocks by using a reference selected from the first reference of the first data object and second reference of the second data object.

2. The method of claim 1, further comprising:

recovering deduplicated data blocks represented by the first data object by using the second data object referenced by the first reference included in the first data object.

3. The method of claim 1, further comprising:

recovering deduplicated data blocks represented by the second data object by using the first data object referenced by the second reference included in the second data object.

4. The method of claim 1, wherein a data object is selected from the group consisting of a deduplication domain, a storage extent, a LUN, a file and a data block, wherein the data block is a fixed size chunk of physical disk storage.

5. The method of claim 4, wherein the deduplication domain comprises a set of storage extents, wherein each storage extent of the set of storage extents comprises a set of LUNs, each LUN of the set of LUNs is a logical representation of a subset of physical disk storage.

6. The method of claim 1, wherein a digest is associated with a data object, wherein the digest is based on contents of the data object and uniquely identifies the data object.

7. The method of claim 1, wherein an index table comprises a set of index entries, wherein each index entry of the set of index entries includes a deduplication density of a data object and is associated with a digest of the data object.

8. The method of claim 1, further comprising:

comparing the deduplication density of the first data object with a threshold value.

9. The method of claim 1, wherein applying the deduplicating technique further comprising:

computing a digest for a data object; and

processing the digest for the data object.

10. A system for use in managing deduplication density in a storage system, the system comprising:

a processor;

first logic determining a deduplication density of a first data object, wherein the deduplication density is indicated by a number of deduplicated data objects represented by the first data object;

second logic skipping, based on the deduplication density of the first data object, the first data object for applying a deduplicating technique to the first data object;

third logic creating an association between the first data object and a second data object, wherein the second data object is used for applying the deduplicating technique, wherein data stored in the first data object is identical to data stored in the second data object, wherein creating the association between the first and second data objects include adding a first reference in the first data object pointing to the second data object and adding a second reference in the second data object pointing to the first data object; and

fourth logic recovering a set of deduplicated data blocks by using a reference selected from the first reference of the first data object and second reference of the second data object.

11. The system of claim 10, further comprising:

fifth logic recovering deduplicated data blocks represented by the first data object by using the second data object referenced by the first reference included in the first data object.

12. The system of claim 10, further comprising:

fifth logic recovering deduplicated data blocks represented by the second data object by using the first data object referenced by the second reference included in the second data object.

13. The system of claim 10, wherein a data object is selected from the group consisting of a deduplication domain, a storage extent, a LUN, a file and a data block, wherein the data block is a fixed size chunk of physical disk storage.

14. The system of claim 13, wherein the deduplication domain comprises a set of storage extents, wherein each storage extent of the set of storage extents comprises a set of LUNs, each LUN of the set of LUNs is a logical representation of a subset of physical disk storage.

15. The system of claim 10, wherein a digest is associated with a data object, wherein the digest is based on contents of the data object and uniquely identifies the data object.

16. The system of claim 10, wherein an index table comprises a set of index entries, wherein each index entry of the set of index entries includes a deduplication density of a data object and is associated with a digest of the data object.

17. The system of claim 10, further comprising:

comparing the deduplication density of the first data object with a threshold value.

18. The system of claim 10, wherein applying the deduplicating technique further comprising:

computing a digest for a data object; and

processing the digest for the data object.