Optimization of a Partially Deduplicated File

• US 20130060739A1
• Filed: 09/01/2011
• Published: 03/07/2013
• Est. Priority Date: 09/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kalach, Ran, Oltean, Paul Adrian, Benton, James Robert, Gupta, Abhishek, Hasan, Kashif, Cheung, Chun Ho
• Granted Patent: US 8,990,171 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 16/1752 based on file chunks

Abstract

The subject disclosure is directed towards transforming a file having at least one undeduplicated portion into a fully deduplicated file. For each of the at least one undeduplicated portion, a deduplication mechanism defines at least one chunk between file offsets associated with the at least one undeduplicated portion. Chunk boundaries associated with the at least one chunk are stored within deduplication metadata. The deduplication mechanism aligns the at least one chunk with chunk boundaries of at least one deduplicated portion of the file. Then, the at least one chunk is committed to a chunk store.

Claims

1. In a computing environment, a method performed at least in part on at least one processor, comprising, re-optimizing a file having at least one undeduplicated portion into a fully deduplicated file, including processing each undeduplicated portion of the file to determine at least one chunk, committing the at least one chunk to a chunk store, and modifying metadata associated with the file to include a reference to each chunk.

2. The method of claim 1 further comprising combining two undeduplicated portions of the file that are separated by a deduplicated portion into a single undeduplicated portion that includes each undeduplicated portion and the deduplicated portion, before processing that single undeduplicated portion.

3. The method of claim 1 wherein processing at least one undeduplicated portion comprises defining a chunk starting from a file offset that is prior to or equal to a starting file offset associated with that undeduplicated portion.

4. The method of claim 3 further comprising, determining the file offset by aligning with a last chunk fully contained in a preceding deduplicated portion.

5. The method of claim 1 wherein processing at least one undeduplicated portion comprises defining a chunk by ending chunking of that portion at a file offset that is subsequent to or equal to an ending file offset associated with that undeduplicated portion.

6. The method of claim 5 further comprising determining the ending file offset based upon where the chunking of the undeduplicated portion aligns with one or more existing chunks associated with a preceding deduplicated portion.

7. The method of claim 5 wherein chunking continues beyond the end of that undeduplicated portion, and further comprising, determining whether chunk alignment is achieved within a size-based threshold or chunk number-based threshold, and if alignment is not achieved within the threshold, stopping chunking.

8. The method of claim 1, further comprising, partitioning the file into a plurality of sub-file partitions, and wherein re-optimizing the file comprises separately processing each sub-file partition.

9. The method of claim 1 wherein modifying the metadata occurs in conjunction with committing at least one re-optimized chunk, and wherein modifying the metadata includes combining chunk definitions of deduplicated portions with chunk definitions of the at least one re-optimized chunk.

10. The method of claim 1 further comprising pausing, and subsequently resuming, the re-optimizing of the file.

11. The method of claim 10 further comprising, committing at least one chunk to the chunk store before pausing.

12. The method of claim 11 wherein committing the at least one chunk to the chunk store and pausing occurs based upon one or more criteria, the criteria including file size, when a machine performing the re-optimization becomes non idle, or when the machine is going to be shutdown.

13. The method of claim 11, further comprising, maintaining metadata that tracks which portion of the file is deduplicated, and truncating the portion of the file that is deduplicated.

14. The method of claim 1 wherein the chunks are committed to a chunk store, and further comprising, garbage collecting at least one chunk from the chunk store that is no longer referenced as a result of re-optimizing the file.

15. In a computing environment, a system, comprising, a deduplication mechanism configured to re-optimize a file having at least one undeduplicated portion into a fully deduplicated file, the deduplication mechanism configured to chunk each undeduplicated portion into at least one re-optimized chunk, to commit at least one re-optimized chunk into a chunk store prior to fully re-optimizing the file, and to modify file metadata that represents each chunk of the file that is committed.

16. The system of claim 15, wherein the deduplication mechanism partitions the file into sub-file partitions, commits each re-optimized chunk of each sub-file partition before beginning re-optimizing of a next sub-file partition or when the last sub-file partition has been re-optimized, and truncates the file based upon which sub-file partition has been re-optimized.

17. The system of claim 15 wherein the deduplication mechanism commits the at least one re-optimized chunk into the chunk store prior to fully re-optimizing the file based upon a pause instruction.

18. The system of claim 17, wherein the deduplication mechanism resumes re-optimizing the file after the pause instruction based upon a resume instruction.

19. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

defining a chunk for an undeduplicated portion of a file;

aligning chunk boundaries of the chunk with file offsets associated with at least one deduplicated portion of the file;

updating metadata associated with the file based upon the chunk boundaries; and

committing the chunk to a chunk store.

20. The one or more computer-readable media of claim 19 having further computer-executable instructions comprising, combining two undeduplicated portions of the file that are separated by a deduplicated portion into a single undeduplicated portion that includes each undeduplicated portion and the deduplicated portion.