Transparent virtualization of cloud storage

• US 9,830,271 B2
• Filed: 07/25/2012
• Issued: 11/28/2017
• Est. Priority Date: 07/25/2012
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Phelan, Thomas A., Cota-Robles, Erik, Barry, David William, Back, Adam
• Primary Examiner(s): Bragdon, Reginald
• Assistant Examiner(s): VO, THANH DUC
• Application Number:
  US13/557,650
• Publication Number:
  US 20140032850A1
• Time in Patent Office: 1,952 Days
• CPC Class Codes: G06F 12/0873 Mapping of cache memory to specific storage devices or parts thereof
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 2212/311 In host system

Abstract

Embodiments present a virtual disk image to applications such as virtual machines (VMs) executing on a computing device. The virtual disk image corresponds to one or more subparts of binary large objects (blobs) of data stored by a cloud service, and is implemented in a log structured format. Grains of the virtual disk image are cached by the computing device. The computing device caches only a subset of the grains and performs write operations without blocking the applications to reduce storage latency perceived by the applications. Some embodiments enable the applications that lack enterprise class storage to benefit from enterprise class cloud storage services.

Claims

1. A host computing device comprising:

a local memory including a memory area caching a subset of a plurality of grains corresponding to a binary large object (blob) representing a virtual disk image for a virtual computing instance running in the host computing device, the grains including a first grain, the blob having a plurality of subparts stored by a cloud service connected to the host computing device via a network; and

a processor programmed to;

present the blob to the virtual computing instance for access by the virtual computing instance as the virtual disk image for the virtual computing instance;

define a grain map that is stored in the local memory and correlates the plurality of grains of the virtual disk image to the plurality of subparts of the blob;

receive, from the virtual computing instance, a data operation directed to the first grain;

determine whether or not the first grain is cached in the memory area;

if the first grain is cached in the memory area, perform the received data operation on the memory area for said first grain without communicating via the network;

if the first grain is not cached in the memory area, identify via the defined grain map that is stored in the local memory a subpart of the blob corresponding to the first grain, obtain the first grain from the subpart of the blob via the network, and perform the received data operation on the first grain obtained from the subpart of the blob via the network; and

confirm completion of the data operation to the virtual computing instance,

wherein the data operation comprises a data read operation or a data write operation, and

wherein the processor is further programmed to;

(i) confirm completion of the data write operation to said virtual computing instance before completing the received data write operation on the first grain to reduce latency perceived by said virtual computing instance; and

(ii) perform the data read operation and evaluate partial reads for early return to said virtual computing instance.

2. The host computing device of claim 1,

wherein the processor is programmed to receive a plurality of data operations, and

wherein the processor is further programmed to implement the virtual disk image in a log structured format to perform the received plurality of data operations sequentially.

3. The host computing device of claim 1, wherein the plurality of grains correspond to an address space of the virtual disk image.

4. The host computing device of claim 1, wherein, on receiving a plurality of read requests directed to reserved mappings in the grain map, the processor is programmed to synthesize, without communicating with the cloud service, at least one common pattern for said one or more of the plurality of grains not cached in the memory area.

5. The host computing device of claim 1, wherein, on receiving a plurality of data write operations directed to at least one common pattern of one or more of the plurality of grains, the processor is further programmed to update the grain map with a reserved mapping without sending any data to the cloud service.

6. The host computing device of claim 1, wherein the processor is further programmed to:

detect a frequently occurring pattern for one or more of the plurality of grains in the received data operation; and

map the detected pattern to the same grain in the same subpart of the blob.

7. A method comprising:

subdividing, by a host computing device, a binary large object (blob) of data stored by a cloud service into a plurality of subparts, the blob being presented as a virtual disk image to a virtual computing instance executing on the host computing device for access by the virtual computing instance as the virtual disk image for the virtual computing instance and storing the virtual disk image in the subparts in units of grains;

defining a grain map that is stored in a local memory of the host computing device and correlates the grains of the virtual disk image to the subparts of the blob;

receiving, from the virtual computing instance, a data operation directed to a first grain of the virtual disk image;

determining whether or not the first grain is stored in a local cache of the host computing device;

if the first grain is stored in the local cache, performing the received data operation on the local cache for said first grain without communicating with the cloud service;

if the first grain is not stored in the local cache, identifying via the defined grain map a subpart of the blob corresponding to the first grain, obtaining the first grain from the subpart of the blob, and performing the received data operation on the first grain obtained from the subpart of the blob; and

confirming completion of the data operation to the virtual computing instance,

wherein the data operation comprises a data read operation or a data write operation, and the method further comprises;

(i) confirming completion of the data write operation to the virtual computing instance before completing the received data write operation on the first grain to reduce latency perceived by the virtual computing instance; and

(ii) performing the data read operation and evaluating partial reads for early return to the virtual computing instance.

8. The method of claim 7, further comprising:

receiving a plurality of data operations over time; and

storing, by the host computing device in the local cache for a predefined duration, only the grains corresponding to the subparts of the blob affected by the received plurality of data operations.

9. The method of claim 7, wherein performing the received data operation on the local cache comprises performing the received data operation without encryption, and further comprising encrypting only data sent to the cloud service.

10. The method of claim 7, wherein the local cache of the host computing device is backed by the local memory of the host computing device.

11. One or more computer-readable storage media including computer-executable instructions that, when executed, cause at least one processor of a host computing device to present a virtual disk image from a cloud service to a virtual computing instance running in the host computing device for access by the virtual computing instance as the virtual disk image for the virtual computing instance, while minimizing transfer costs, by performing a method comprising:

accessing a binary large object (blob) of data stored by the cloud service, the blob being subdivided by the host computing device into a plurality of subparts, the blob being presented as a virtual disk image to the virtual computing instance and storing the virtual disk image in the subparts in units of grains,

defining a grain map that is stored in a local memory of the host computing device and correlates the grains of the virtual disk image to the subparts of the blob;

receiving, from the virtual computing instance, a data operation directed to a first grain of the virtual disk image;

determining whether or not the first grain is stored in a local cache of the host computing device;

if the first grain is stored in the local cache, performing the received data operation on the local cache for said first grain without communicating with the cloud service;

if the first grain is not stored in the local cache, for both a read data operation that is received and a write data operation that is received, identifying via the defined grain map a subpart of the blob corresponding to the first grain, obtaining the first grain from the subpart of the blob, and performing the received data operation on the first grain obtained from the subpart of the blob; and

confirming completion of the data operation to the virtual computing instance,

wherein the data operation comprises a data read operation or a data write operation, and the method further comprises;

(i) confirming completion of the data write operation to the virtual computing instance before completing the received data write operation on the first grain to reduce latency perceived by the virtual computing instance; and

(ii) performing the data read operation and evaluating partial reads for early return to the virtual computing instance.

12. The computer storage media of claim 11, wherein the computer-executable instructions further cause the processor to periodically flush the local cache to the cloud service.

13. The computer storage media of claim 11, wherein the computer-executable instructions further cause the processor to map virtual disk image access by the virtual computing instance to blob access by the host computing device.

14. The computer storage media of claim 11, wherein the computer-executable instructions cause the processor to minimize I/O transfers by storing in the local cache only the grains accessed by the virtual computing instance.

15. The computer storage media of claim 11, wherein the local cache of the host computing device is backed by the local memory of the host computing device.