Real time cloud bursting

• US 9,753,669 B2
• Filed: 11/26/2014
• Issued: 09/05/2017
• Est. Priority Date: 05/13/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Velostrata, Ltd. (Alphabet Inc.)
• Inventors: Ben-Shaul, Israel, Degany, Ady, Vasetsky, Leonid, Glixman, Shahar, Ben-Ari, Yaniv
• Primary Examiner(s): Brown, Michael J
• Application Number:
  US14/554,837
• Publication Number:
  US 20150331635A1
• Time in Patent Office: 1,014 Days
• Field of Search: 713 1, 713 2, 713100, 713300
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5088 involving task migration
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/2876 Pairs of inter-processing entities at each side of the network, e.g. split proxies
• CPC Class Codes: H04L 67/5681 Pre-fetching or pre-delivering data based on network characteristics
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

A method and system for real-time cloud bursting is provided. The method and system are directed to extending a data center with cloud computing resources by decoupling computing resources and storage devices in a virtualized data center, and booting the decoupled computing resources in a staged process while storage devices are divided and prioritized into components. Data and boot instructions are re-routed and cached as needed through a proxy system.

Claims

1. A method comprising:

maintaining a local storage resource associated with a computing instance running remotely, wherein the instance couples to the local storage resource through a first edge appliance and a second edge appliance;

streaming prioritized blocks from the local storage resource through the first edge appliance to a cache in the second edge appliance;

requesting, from the second edge appliance to the first edge appliance, blocks called by the instance, but not yet streamed to the second edge appliance from the first edge appliance;

wherein the second edge appliance serves as a block storage device for the instance;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, further comprising determining a block priority by:

identifying a group of files accessed by a workload selected to run on the instance;

sorting the group of files;

for each file in the group of files,

mapping a file to a set of corresponding blocks,

setting a block priority of each block in the set based on the sorting the group of files.

3. The method of claim 1, further comprising determining a block priority by:

identifying a set of blocks accessed by a workload selected to run on the instance;

sorting the set of blocks in the local storage resource by most accessed to least accessed;

setting a block priority of each block in the set based on the sorting.

4. The method of claim 1, wherein the first edge appliance runs on a first set of hardware from a first computer through software selected from a group comprising:

a first operating system on a first virtual machine connected to the first set of hardware through a first type one hypervisor,

the first operating system on the first virtual machine connected to the first set of hardware through a first type two hypervisor, and

the first operating system connected directly to the first set of hardware;

wherein the second edge appliance runs on a second set of hardware from a second computer through software selected from a group comprising;

a second operating system on a second virtual machine connected to the second set of hardware through a second type one hypervisor,

the second operating system on the second virtual machine connected to the second set of hardware through a second type two hypervisor, and

the second operating system connected directly to the second set of hardware.

5. A system comprising:

a local storage resource;

a first edge appliance, coupled to the local storage resource;

a second edge appliance, remotely coupled to the first edge appliance;

a computing instance associated with an executing workload and associated with the local storage resource, coupled to the second edge appliance;

reading logic, in the second edge appliance, that is configured to maintain reads for the executing workload by, streaming prioritized blocks from the local storage resource through the first edge appliance to a secure cache in the second edge appliance;

wherein the reading logic is configured to maintain reads of the executing workload by requesting, from the second edge appliance to the first edge appliance, blocks called by the instance, and not yet streamed to the second edge appliance from the first edge appliance;

wherein the second edge appliance serves as a block storage device for the instance.

6. The system of claim 5, further comprising writing logic, in the second edge appliance, that is configured to maintain writes for the executing workload by aggregating written blocks in the second edge appliance to a threshold amount, and send the aggregated written blocks, from the second edge appliance through the first edge appliance, to the local storage resource.

7. The system of claim 6, wherein the writing logic, in the second edge appliance, that is configured to maintain writes for the executing workload, upon an instruction to send the instance running remotely back to a local computing device, immediately flushes any aggregated written blocks to the local storage resource.

8. The system of claim 5, further comprising writing logic, in the second edge appliance, that is configured to maintain writes for the executing workload by receiving a written block from the instance;

storing the written block in a remote storage resource; and

routing requests for data contained in the written block through the second edge appliance to the remote storage resource.

9. The system of claim 5, wherein the local storage resource comprises a virtual disk.

10. A method comprising:

executing a workload on a computer associated with a first operating system and a local storage resource;

coupling a first edge appliance to the computer;

remotely coupling a second edge appliance to the first edge appliance;

wherein the second edge appliance contains a diskless boot sequence and a cache that is empty;

wherein the cache is configured to store a partial image of the first operating system from the computer for future boot sequences;

wherein the first edge appliance is configured to stream blocks from the local storage resource to store in the cache of the second edge appliance.

11. The method of claim 10, wherein the executing workload is disassociated from the computer and is executed remotely by performing the further steps of:

using the first edge appliance, taking a snapshot of the computer;

using the first edge appliance, decoupling the local storage resource from the computer by routing traffic to and from the local storage resource through the first edge appliance;

using the first edge appliance, sending a message to provision a remote server coupled to the second edge appliance and configured with the diskless boot sequence;

using the remote server, performing a diskless boot of a computing instance by requesting a set of operating system blocks from the second edge appliance, which may be served using the partial image of the first operating system or further requested from the first edge appliance to be served from a portion of the snapshot;

using the remote server, mapping at least one mount point of the instance to the second edge appliance, wherein the second edge appliance serves as a block storage device for the instance;

using the first edge appliance, streaming, to the second edge appliance, a set of blocks contained in the portion of the snapshot, enabling the instance to finish booting and to run the executing workload.

12. The method of claim 11, wherein the message to provision the remote server includes a set of specifications;

wherein the set of specifications are determined by performing the further steps of;

using the first edge appliance, scanning the snapshot;

wherein the scanning determines resources needed by the computer including number of CPUs, amount of RAM, and number of input/output operations per second;

wherein the set of specifications includes the resources needed to ensure the remote server comprises the resources necessary to run the instance.

13. The method of claim 11, wherein the executing workload is disassociated from the computer and is executed remotely by performing the further steps of:

using the second edge appliance, requesting, from the first edge appliance, blocks called by the instance, but not yet streamed to the second edge appliance from the first edge appliance.

14. The method of claim 11, wherein the second edge appliance contains a resizable system cache;

wherein the resizable system cache in the second edge appliance serves as the block storage device for the instance;

the resizable system cache comprising a first size cache accessed during booting and a second size cache accessed while running the executing workload.

15. The method of claim 11, wherein the first edge appliance runs on a first set of hardware from a first computer through software selected from a group comprising:

a second operating system on a first virtual machine connected to the first set of hardware through a type one hypervisor,

the second operating system on the first virtual machine connected to the first set of hardware through a type two hypervisor, and

the second operating system connected directly to the first set of hardware;

wherein the second edge appliance runs on a second set of hardware from a second computer through software selected from a group comprising;

a third operating system on a second virtual machine connected to the second set of hardware through the type one hypervisor,

the third operating system on the second virtual machine connected to the second set of hardware through the type two hypervisor, and

the third operating system connected directly to the second set of hardware.

16. The method of claim 11, wherein the executing workload is re-associated with the computer after disassociating from the computer by performing the steps of:

flushing any pending writes to the local storage resource;

booting the computer from the snapshot of the computer.

17. The method of claim 11, wherein the executing workload is disassociated from the computer and is executed remotely by performing the further steps of:

providing a migration command for mirroring the local storage resource to a remote storage resource after booting and running the first operating system for the executing workload on the remote server.