MAINTAINING DEDUPLICATION DATA IN NATIVE FILE FORMATS

• US 20130080404A1
• Filed: 09/23/2011
• Published: 03/28/2013
• Est. Priority Date: 09/23/2011
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Smith, Brian, Ramsdell, Tom, Rodriquez, Rob, Shintre, Shweta
• Granted Patent: US 9,442,951 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1752 based on file chunks

Abstract

Mechanisms are provided to maintain deduplication data in native file formats. Files, including entities such as volumes and databases, are analyzed to identify components suitable for deduplication. These components suitable for deduplication are delineated into chunks and identifiers are generated for each of the chunks. The identifiers are used to reference the chunks in deduplication dictionaries that provide locations indicating where deduplicated chunks are stored. The components in the files are replaced with file handles or stubs that applications can use to access deduplicated data. Applications can continue to perform operations on the files as though no deduplication has occurred.

Claims

1. A method, comprising:

parsing a file to identify a first plurality of components including a first component and a second component of the file;

replacing the first component in the file with a first stub and the second component in the file with a second stub;

delineating the first component into a first plurality of chunks;

generating a first chunk identifier corresponding to a first chunk, the first chunk identifier used to access a deduplication dictionary;

determining whether the first chunk is already stored in a deduplication system using the first chunk identifier and the deduplication dictionary.

2. The method of claim 1, wherein the first component is further separated into a first plurality of subcomponents.

3. The method of claim 1, wherein the file is maintained by an application in an application native file format.

4. The method of claim 3, wherein the first stub is accessed by the application without adversely affecting application operation.

5. The method of claim 4, wherein the first chunk identifier is a hash of the first chunk.

6. The method of claim 3, wherein the deduplication dictionary maintains a plurality of chunk identifiers and corresponding chunk locations.

7. The method of claim 6, wherein the first chunk is maintained in a datastore suitcase.

8. The method of claim 1, wherein if the first chunk is already stored in the deduplication system, reference counts associated with the datastore suitcase are updated.

9. The method of claim 1, wherein if the first chunk is not already stored in the deduplication system, a new entry is created in the deduplication dictionary and the first chunk is stored in a datastore suitcase.

10. The method of claim 1, wherein the second component is further separated into a second plurality of subcomponents.

11. A computer readable medium, comprising:

computer code for parsing a file to identify a first plurality of components including a first component and a second component of the file;

computer code for replacing the first component in the file with a first stub and the second component in the file with a second stub;

computer code for delineating the first component into a first plurality of chunks;

computer code for generating a first chunk identifier corresponding to a first chunk, the first chunk identifier used to access a deduplication dictionary;

computer code for determining whether the first chunk is already stored in a deduplication system using the first chunk identifier and the deduplication dictionary.

12. The computer readable medium of claim 11, wherein the first component is further separated into a first plurality of subcomponents.

13. The computer readable medium of claim 11, wherein the file is maintained by an application in an application native file format.

14. The computer readable medium of claim 13, wherein the first stub is accessed by the application without adversely affecting application operation.

15. The computer readable medium of claim 14, wherein the first chunk identifier is a hash of the first chunk.

16. The computer readable medium of claim 13, wherein the deduplication dictionary maintains a plurality of chunk identifiers and corresponding chunk locations.

17. The computer readable medium of claim 16, wherein the first chunk is maintained in a datastore suitcase.

18. The computer readable medium of claim 11, wherein if the first chunk is already stored in the deduplication system, reference counts associated with the datastore suitcase are updated.

19. The computer readable medium of claim 11, wherein if the first chunk is not already stored in the deduplication system, a new entry is created in the deduplication dictionary and the first chunk is stored in a datastore suitcase.

20. A method, comprising:

parsing a file to identify a first plurality of components including a first component and a second component of the file;

replacing the first component in the file with a first stub and the second component in the file with a second stub;

delineating the first component into a first plurality of chunks;

generating a first chunk identifier corresponding to a first chunk, the first chunk identifier used to access a deduplication dictionary;

determining whether the first chunk is already stored in a deduplication system using the first chunk identifier and the deduplication dictionary.