ELASTIC TEMPORARY FILESYSTEM

• US 20150160884A1
• Filed: 10/17/2014
• Published: 06/11/2015
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: SCALES, Daniel J., MAGDON-ISMAIL, Tariq, CHEVERESAN, Razvan, NELSON, Michael, MCDOUGALL, Richard
• Granted Patent: US 10,241,709 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

An elastic filesystem for temporary data provides storage space for virtual machines (VMs) in a distributed computing system. The filesystem redirects accesses to virtual disks in VMs to a common pool file. The system provides performance and storage efficiency at least on par with local, direct attached virtual disks, while providing a single pool of shared storage that is provisioned and managed independently of the VMs. The system provides storage isolation between VMs storing temporary data in that shared pool. Also, storage space for temporary data may be allocated on demand and reclaimed when no longer needed, thereby supporting a wide variety of temporary space requirements for different Hadoop jobs.

Claims

1. A method comprising:

intercepting, by operation of an I/O virtualization layer, a storage command associated with a first virtual machine (VM), wherein the storage command specifies a logical storage space of a virtual disk;

determining a location of an extent corresponding to the logical storage space based on a mapping comprising a list of contiguous extents within a pool file that have been written to the virtual disk; and

executing the storage command on the extent at the determined location within the pool file.

2. The method of claim 1, wherein executing the storage command on the extent at the determined location within the pool file further comprises:

executing a write operation of temporary data to the extent corresponding to the logical storage space.

3. The method of claim 1, wherein the storage command comprises a write operation of temporary data to the logical storage space, wherein the method further comprises:

responsive to determining no extent corresponds to the logical storage space, allocating a new extent from the pool file;

updating the mapping comprising a location of the allocated extent within the pool file and the associated first VM; and

writing the temporary data to the allocated extent based on the mapping.

4. The method of claim 3, wherein updating the mapping comprising the location of the extent within the pool file and the associated first VM further comprises:

determining the allocated extent is located within the pool file adjacent to an existing extent associated with the first VM; and

updating the mapping comprising a location of the allocated extent and the existing extent as a single contiguous extent within the pool file and the associated first VM.

5. The method of claim 3, wherein the allocating the new extent from the pool file further comprises:

responsive to determining a size of the temporary data of the write operation exceeds a threshold size, allocating the new extent having a first size; and

responsive to determining the size of the temporary data of the write operation does not exceed the threshold size, allocating the new extent having a second size.

6. The method of claim 1, wherein executing the storage command on the extent at the determined location within the pool file further comprises:

executing a read operation of temporary data from the extent corresponding to the logical storage space.

7. The method of claim 1, wherein the storage command comprises an unmap operation indicating temporary data contained in the logical storage space has been removed;

wherein executing the storage command on the extent at the determined location within the pool file further comprises;

de-allocating the extent corresponding to the logical storage space.

8. The method of claim 1, further comprising:

responsive to powering off the first VM, de-allocating a plurality of extents associated with the first VM from the pool file.

9. A non-transitory computer-readable storage medium comprising instructions that, when executed in a computing device, perform the steps of:

intercepting, by operation of an I/O virtualization layer, a storage command associated with a first virtual machine (VM), wherein the storage command specifies a logical storage space of a virtual disk;

determining a location of an extent corresponding to the logical storage space based on a mapping comprising a list of contiguous extents within a pool file that have been written to the virtual disk; and

executing the storage command on the extent at the determined location within the pool file.

10. The non-transitory computer-readable storage medium of claim 9, wherein executing the storage command on the extent at the determined location within the pool file further comprises:

executing a write operation of temporary data to the extent corresponding to the logical storage space.

11. The non-transitory computer-readable storage medium of claim 9, wherein the storage command comprises a write operation of temporary data to the logical storage space, wherein the method further comprises:

responsive to determining no extent corresponds to the logical storage space, allocating a new extent from the pool file;

updating the mapping comprising a location of the allocated extent within the pool file and the associated first VM; and

writing the temporary data to the allocated extent based on the mapping.

12. The non-transitory computer-readable storage medium of claim 11, wherein updating the mapping comprising the location of the extent within the pool file and the associated first VM further comprises:

determining the allocated extent is located within the pool file adjacent to an existing extent associated with the first VM; and

updating the mapping comprising a location of the allocated extent and the existing extent as a single contiguous extent within the pool file and the associated first VM.

13. The non-transitory computer-readable storage medium of claim 11, wherein the allocating the new extent from the pool file further comprises:

responsive to determining a size of the temporary data of the write operation exceeds a threshold size, allocating the new extent having a first size; and

responsive to determining the size of the temporary data of the write operation does not exceed the threshold size, allocating the new extent having a second size.

14. The non-transitory computer-readable storage medium of claim 9, wherein executing the storage command on the extent at the determined location within the pool file further comprises:

executing a read operation of temporary data from the extent corresponding to the logical storage space.

15. The non-transitory computer-readable storage medium of claim 9, wherein the storage command comprises an unmap operation indicating temporary data contained in the logical storage space has been removed;

wherein executing the storage command on the extent at the determined location within the pool file further comprises;

de-allocating the extent corresponding to the logical storage space.

16. The non-transitory computer-readable storage medium of claim 9, wherein steps further comprise:

responsive to powering off the first VM, de-allocating a plurality of extents associated with the first VM from the pool file.

17. A computer system comprising:

a storage device having a pool file comprising a first plurality of extents associated with a first virtual machine (VM) and a second plurality of extents associated with a second VM;

a system memory having a mapping of the first and second plurality of extents that have been written to a first virtual disk and a second virtual disk; and

a processor programmed to carry out the steps of;

intercepting a storage command associated with the first VM, wherein the storage command specifies a logical storage space of the first virtual disk;

determining a location of a first extent of the first plurality of extents corresponding to the logical storage space based on the mapping; and

executing the storage command on the first extent at the determined location within the pool file.

18. The computer system of claim 17, wherein the processor is further programmed to execute the storage command on the first extent at the determined location within the pool file further comprises:

executing a write operation of temporary data to the first extent corresponding to the logical storage space.

19. The computer system of claim 17, wherein the storage command comprises a write operation of temporary data to the logical storage space, wherein the processor is further programmed to carry out the steps of:

responsive to determining no extent corresponds to the logical storage space, allocating a new extent from the pool file;

updating the mapping comprising a location of the allocated extent within the pool file and the associated first VM; and

writing the temporary data to the allocated extent based on the mapping.

20. The computer system of claim 17, wherein the storage command comprises an unmap operation indicating temporary data contained in the logical storage space has been removed, wherein the processor programmed to execute the storage command on the first extent at the determined location within the pool file is further programmed to carry out the steps of:

de-allocating the first extent corresponding to the logical storage space.