Low latency computational capacity provisioning

• US 9,678,773 B1
• Filed: 09/30/2014
• Issued: 06/13/2017
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Thomas, Dylan Chandler, Wagner, Timothy Allen, Reque, Sean Philip
• Primary Examiner(s): Dao, Thuy
• Application Number:
  US14/502,810
• Time in Patent Office: 987 Days
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 9/44521 Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45545 Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A system for providing low latency computational capacity is provided. The system may be configured to maintain a pool of virtual machine instances, which may be assigned to users to service the requests associated with the users. The system may further be configured to receive a request to acquire compute capacity for executing a program code associated with a particular user, determine whether the pool of virtual machine instances includes a container that may be used to execute the program code therein, and cause the program code of the particular user to be executed in the container.

Claims

1. A computer-implemented method comprising:

as implemented by one or more computing devices configured with specific executable instructions,

maintaining an active pool of virtual machine instances assigned to one or more users, wherein each virtual machine instance in the active pool has one or more software components loaded thereon;

receiving a request to acquire compute capacity for executing a program code of a user on the virtual compute system, the request including metadata indicating the program code, the user associated with the program code, and a first amount of computing resources to be allocated for executing the program code on the virtual compute system;

determining whether the active pool contains a container that has the program code loaded thereon;

in response to determining that the active pool does not contain a container that has the program code loaded thereon, determining whether the active pool contains a virtual machine instance that is assigned to the user and has at least the first amount of computing resources available for executing the program code;

in response to determining that the active pool contains a virtual machine instance that is assigned to the user and has at least the first amount of computing resources available for executing the program code, creating a container having at least the first amount of computing resources on the virtual machine instance and assigning the container to handle the request to execute the program code on the virtual compute system; and

causing the program code of the user to be executed in the container created on the virtual machine instance.

2. The computer-implemented method of claim 1, further comprising:

causing the program code of the user to be downloaded onto the container from a storage system logically associated with the virtual machine instance.

3. The computer-implemented method of claim 1, further comprising:

causing the program code of the user to be downloaded onto the virtual machine instance from a web-based data store in network communication with the virtual compute system.

4. The computer-implemented method of claim 1, further comprising maintaining a warming pool of virtual machine instances, wherein the warming pool comprises at least one virtual machine instance that is configured to satisfy at least one of a predetermined plurality of computing constraints specified in the request.

5. A system for providing low-latency computational capacity, the system comprising:

an electronic data store configured to store one or more program codes associated with one or more users; and

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions, said virtual compute system in communication with the data store, and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices, the plurality of virtual machine instances comprising (i) a warming pool comprising unassigned virtual machine instances having one or more software components loaded thereon waiting to be assigned to a user, and (ii) an active pool comprising virtual machine instances assigned to one or more users;

receive a request to execute a program code associated with a particular user of the virtual compute system, the request including metadata indicating information regarding the program code, the particular user associated with the program code, and a first amount of computing resources to be allocated for executing the program code on the virtual compute system;

determine whether the active pool contains a container that has the program code loaded thereon;

in response to determining that the active pool does not contain a container that has the program code loaded thereon, determine whether the active pool contains a virtual machine instance that has the program code stored thereon;

in response to determining that the active pool does not contain a virtual machine instance that has the program code stored thereon, determine whether the active pool contains a virtual machine instance that is assigned to the particular user and has at least the first amount of computing resources available to handle the request;

in response to determining that the active pool does not contain a virtual machine instance that is assigned to the particular user and has at least the first amount of computing resources available to handle the request, select a virtual machine instance from the warming pool;

create a container having at least the first amount of computing resources on the selected virtual machine instance and assign the container to handle the request to execute the program code on the virtual compute system; and

cause the program code associated with the particular user to be loaded and executed in the container.

6. The system of claim 5, wherein the virtual compute system is configured to download the program code associated with the particular user onto the container from the electronic data store.

7. A system, comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain an active pool of virtual machine instances assigned to one or more users, wherein each virtual machine instance in the active pool has one or more software components loaded thereon;

receive a request to acquire compute capacity for executing a program code of a user on the virtual compute system, the request including metadata indicating the program code, the user associated with the program code, and a first amount of computing resources to be allocated for executing the program code on the virtual compute system;

determine whether the active pool contains a container that has the program code loaded thereon;

in response to determining that the active pool does not contain a container that has the program code loaded thereon, determine whether the active pool contains a virtual machine instance that is assigned to the user and has at least the first amount of computing resources available for executing the program code;

in response to determining that the active pool contains a virtual machine instance that is assigned to the user and has at least the first amount of computing resources available for executing the program code, create a container having at least the first amount of computing resources on the virtual machine instance and assign the container to handle the request to execute the program code on the virtual compute system; and

cause the program code of the user to be executed in the container created on the virtual machine instance.

8. The system of claim 7, wherein the virtual compute system is further configured to cause the program code of the user to be downloaded onto the container from a storage system logically associated with the virtual machine instance.

9. The system of claim 7, further comprising a web-based data store in communication with the virtual compute system and configured to store at least the program code of the user, wherein the virtual compute system is further configured to cause the program code of the user to be downloaded onto the container from the web-based data store.

10. The system of claim 7, wherein the request specifies at least one of a predetermined plurality of computing constraints for executing the program code, and wherein the virtual compute system is further configured to maintain a warming pool of virtual machine instances, wherein the warming pool comprises at least one virtual machine instance that is configured to satisfy the at least one of the predetermined plurality of computing constraints specified in the request.

11. The system of claim 7, further comprising an internal code cache configured to store program codes executed on the virtual compute system, wherein the virtual compute system is further configured to store the program code of the user in the internal code cache upon causing the program code of the user to be loaded on the container.

12. The system of claim 7, wherein the request specifies an execution behavior for executing the program code on the virtual compute system, and wherein the virtual compute system is further configured to configure the container based on the execution behavior specified by the request.

13. A computer-readable, non-transitory storage medium storing computer executable instructions that, when executed by one or more computing devices, configure the one or more computing devices to perform operations comprising:

maintaining an active pool of virtual machine instances assigned to one or more users, wherein each virtual machine instance in the active pool has one or more software components loaded thereon;

receiving a request to acquire compute capacity for executing a program code of a user on the virtual compute system, the request including metadata indicating the program code, the user associated with the program code, and a first amount of computing resources to be allocated for executing the program code on the virtual compute system;

determining whether the active pool contains a container that has the program code loaded thereon;

in response to determining that the active pool does not contain a container that has the program code loaded thereon, determining whether the active pool contains a virtual machine instance that is assigned to the user and has at least the first amount of computing resources available for executing the program code;

in response to determining that the active pool contains a virtual machine instance that is assigned to the user and has at least the first amount of computing resources available for executing the program code, creating a container having at least the first amount of computing resources on the virtual machine instance and assigning the container to handle the request to execute the program code on the virtual compute system; and

causing the program code of the user to be executed in the container on the virtual machine instance.

14. The computer-readable, non-transitory storage medium of claim 13, wherein the operations further comprise causing the program code of the user to be downloaded onto the container from a storage system logically associated with the virtual machine instance.

15. The computer-readable, non-transitory storage medium of claim 13, wherein the operations further comprise causing the program code of the user to be downloaded onto the container from a web-based data store in network communication with the virtual compute system.

16. The computer-readable, non-transitory storage medium of claim 13, wherein operations further comprise maintaining a warming pool of virtual machine instances, wherein the warming pool comprises at least one virtual machine instance that is configured to satisfy at least one of a predetermined plurality of computing constraints specified in the request.

17. A system, comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain an active pool of virtual machine instances currently assigned to one or more users, wherein each virtual machine instance in the active pool has one or more software components loaded thereon;

receive a request to acquire compute capacity for executing a program code of a user on the virtual compute system, the request including metadata indicating the program code, the user associated with the program code, and a first amount of computing resources to be allocated for executing the program code on the virtual compute system;

acquire a container having at least the first amount of computing resources, based on an execution priority, for executing the program code of the user, the container created on one of the virtual machine instances in the active pool; and

cause the program code of the user to be executed in the container created on the virtual machine instance.

18. The system of claim 17, wherein the execution priority causes the virtual compute system to determine that there is no container that already has the program code loaded therein at the time the request is received before determining whether the container should be created on a virtual machine instance already assigned to the user at the time the request is received.

19. The system of claim 17, wherein the execution priority causes the virtual compute system to determine that there is no virtual machine instance already assigned to the user at the time the request is received and having at least the first amount of computing resources before determining whether the container should be created on a virtual machine instance not yet created or assigned to the user at the time the request is received.

20. The system of claim 17, wherein the request specifies at least one of a predetermined plurality of computing constraints for executing the program code, and wherein the virtual compute system is further configured to maintain a warming pool of virtual machine instances, wherein the warming pool comprises at least one virtual machine instance that is configured to satisfy the at least one of the predetermined plurality of computing constraints specified in the request.