Data Deduplication in a Virtualization Environment

• US 20120151177A1
• Filed: 12/14/2010
• Published: 06/14/2012
• Est. Priority Date: 12/14/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kalach, Ran, Oltean, Paul Adrian, Teodorescu, Cristian G., Dickson, Mathew James
• Granted Patent: US 8,959,293 B2
• Time in Patent Office: Days
• US Class Current: 711/203
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Techniques are described herein that are capable of optimizing (i.e., deduplicating) data in a virtualization environment. For example, optimization designations (a.k.a. deduplication designations) may be assigned to respective regions of a virtualized storage file. A virtualized storage file is a file that is configured to be mounted as a disk or a volume to provide a file system interface for accessing hosted files. In accordance with this example, each optimization designation indicates an extent to which the respective region is to be optimized (i.e., deduplicated). In another example, a virtualized storage file is mounted to provide a virtual disk that includes hosted files. In accordance with this example, optimization designations are assigned to the respective hosted files. In further accordance with this example, each optimization designation indicates an extent to which the respective hosted file is to be optimized.

Claims

1. A method comprising:

mounting a virtualized storage file to provide a virtual disk that includes a plurality of hosted files;

mapping each hosted file of the plurality of hosted files to one or more of a plurality of regions of the virtualized storage file;

assigning a plurality of optimization designations to the plurality of respective regions, each optimization designation being assigned to the respective region based on at least one property of the hosted file that is mapped to that region, each optimization designation indicating an extent to which the respective region is to be optimized; and

optimizing each region to the extent that is indicated by the respective optimization designation that is assigned to that region.

2. The method of claim 1, wherein mapping each hosted file of the plurality of hosted files to one or more regions comprises:

mounting a snapshot of the virtualized storage file to provide a volume that is arranged in accordance with a file system that indicates a plurality of disk offsets that corresponds to the plurality of respective hosted files;

analyzing the volume to determine the plurality of disk offsets; and

mapping the plurality of disk offsets to a plurality of respective virtualized storage file offsets in the virtualized storage file, the plurality of virtualized storage file offsets corresponding to respective regions in the plurality of regions.

3. The method of claim 1, wherein mapping each hosted file of the plurality of hosted files to one or more regions comprises:

reviewing an on-disk format of a file system that is associated with the virtual disk to obtain metadata that indicates that each hosted file of the plurality of hosted files corresponds to one or more of a plurality of virtualized storage file offsets in the virtualized storage file, the plurality of virtualized storage file offsets corresponding to the plurality of respective regions of the virtualized storage file.

4. The method of claim 1, wherein mounting the virtualized storage file comprises:

mounting the virtualized storage file on a virtual machine that is executing on host device to provide the virtual disk; and

wherein optimizing each region comprises;

optimizing each region by the host device.

5. The method of claim 4, wherein mapping each hosted file of the plurality of hosted files to one or more regions comprises:

mapping each hosted file of the plurality of hosted files to one or more regions of the virtualized storage file by the virtual machine using a file system application programming interface and a volume application programming interface;

wherein the method further comprises;

providing an optimization indicator from the virtual machine to the host device, the optimization indicator specifying the plurality of optimization designations and including information regarding the mapping; and

wherein optimizing each region comprises;

optimizing each region by the host device based on the optimization indicator.

6. A method comprising:

assigning a plurality of optimization designations to a plurality of respective regions of a virtualized storage file, the plurality of regions corresponding to a plurality of respective virtualized storage file offsets in the virtualized storage file, each optimization designation being assigned to the respective region based on at least one property of a hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region, each optimization designation indicating an extent to which the respective region is to be optimized; and

optimizing each region to the extent that is indicated by the respective optimization designation that is assigned to that region.

7. The method of claim 6, wherein optimizing each region comprises:

optimizing a first region of the plurality of regions to an extent that is indicated by a first optimization designation that is assigned to the first region to provide an optimized representation of the first region; and

wherein the method further comprises;

in response to optimizing the first region, determining one or more changes that occur with respect to the first region since optimizing the first region;

generating a difference file that specifies the one or more changes; and

optimizing the difference file to the extent that is indicated by the first optimization designation without optimizing the optimized representation of the first region to the extent that is indicated by the first optimization designation.

8. The method of claim 7, further comprising:

in response to optimizing the difference file, determining at least one change that occurs with respect to the first region since optimizing the difference file;

generating a second difference file that specifies the at least one change; and

optimizing the second difference file to the extent that is indicated by the first optimization designation without optimizing the optimized representation of the first region to the extent that is indicated by the first optimization designation and without re-optimizing the first difference file to the extent that is indicated by the first optimization designation.

9. The method of claim 6, wherein assigning the plurality of optimization designations comprises:

assigning each optimization designation to the respective region based on at least one of a number of times that the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is accessed or a number of times that the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is modified.

10. The method of claim 6, wherein assigning the plurality of optimization designations comprises:

assigning each optimization designation to the respective region based on at least one of a time at which the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is most recently accessed or a time at which the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is most recently modified.

11. The method of claim 6, wherein assigning the plurality of optimization designations comprises:

assigning each optimization designation to the respective region based on at least one of a frequency with which the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is accessed or a frequency with which the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is modified.

12. The method of claim 6, wherein assigning the plurality of optimization designations comprises:

assigning each optimization designation to the respective region based on a classification of the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region.

13. The method of claim 6, wherein assigning the plurality of optimization designations comprises:

assigning each optimization designation to the respective region based on a format of the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region.

14. The method of claim 6, wherein assigning the plurality of optimization designations comprises:

assigning each optimization designation to the respective region based on whether the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is configured to be used in a system boot operation.

15. A system comprising:

an assignment module configured to assign a plurality of optimization designations to a plurality of respective regions of a virtualized storage file, the plurality of regions corresponding to a plurality of respective virtualized storage file offsets in the virtualized storage file, each optimization designation being assigned to the respective region based on at least one property of a hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region, each optimization designation indicating an extent to which the respective region is to be optimized; and

an optimization module configured to optimize each region to the extent that is indicated by the respective optimization designation that is assigned to that region.

16. The system of claim 15, wherein the optimizer is configured to optimize a first region of the plurality of regions to an extent that is indicated by a first optimization designation that is assigned to the first region to provide an optimized representation of the first region;

wherein the system further comprises;

a determination module configured to determine one or more changes that occur with respect to the first region since optimization of the first region; and

a generation module configured to generate a difference file that specifies the one or more changes; and

wherein the optimizer is further configured to optimize the difference file to the extent that is indicated by the first optimization designation without optimizing the optimized representation of the first region to the extent that is indicated by the first optimization designation.

17. The system of claim 15, wherein the assignment module is configured to assign each optimization designation to the respective region in accordance with a heuristic technique based on the at least one property of the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region.

18. The system of claim 15, wherein the assignment module is configured to assign each optimization designation to the respective region based on at least one of a time at which the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is most recently accessed or a time at which the hosted file that is mapped to the respective virtualized storage file offset that corresponds to that region is most recently modified.

19. The system of claim 15, further comprising:

an offset mapper configured to determine that a plurality of disk offsets that is associated with a plurality of respective segments of the hosted file is changed to a plurality of respective revised disk offsets in accordance with a defragmentation operation, the plurality of disk offsets corresponding to the plurality of respective virtualized storage file offsets, the offset mapper further configured to change a mapping of the plurality of virtualized storage file offsets from the plurality of respective disk offsets to the plurality of respective revised disk offsets.

20. The system of claim 19, wherein the optimization module is configured to modify metadata that indicates an association of the plurality of regions of the virtualized storage file with a plurality of references to a plurality of optimized representations of the plurality of respective regions, in lieu of re-optimizing the plurality of regions of the virtualized storage file to account for the revised disk offsets.