MANAGEMENT OF PERIODIC REQUESTS FOR COMPUTE CAPACITY

• US 20170090961A1
• Filed: 09/30/2015
• Published: 03/30/2017
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wisniewski, Scott Daniel, Wagner, Timothy Allen, Brooker, Marc John
• Granted Patent: US 10,042,660 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2209/5011 Pool
• CPC Class Codes: G06F 2209/5019 Workload prediction
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A system for monitoring incoming code execution requests and scheduling the corresponding code executions is provided. The system may be configured to maintain a plurality of virtual machine instances. The system may be further configured to determine whether at least some of the incoming code execution requests exhibit periodicity, and cause a reduced amount of idle compute capacity to be maintained on the virtual compute system. The system may be further configured to cause additional compute capacity to be added shortly before code execution requests are expected to be received.

Claims

1. A system for providing low-latency computational capacity from a virtual compute fleet, the system comprising:

an electronic data store configured to store at least a program code of a user; and

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions, said virtual compute system in communication with the data store, and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprise;

a warming pool comprising virtual machine instances having one or more software components loaded thereon and waiting to be assigned to a user; and

an active pool comprising virtual machine instances currently assigned to one or more users;

monitor incoming code execution requests to execute program codes on the virtual compute system, at least some of the incoming code execution requests exhibiting a degree of periodicity;

determine the degree of periodicity associated with the at least some of the incoming code execution requests, the determined degree of periodicity indicating a time period at which the at least some of the incoming code execution requests are expected to be received by the virtual compute system;

in response to determining the degree of periodicity associated with the at least some of the incoming code execution requests, cause a reduced number of virtual machine instances to be maintained in the warming pool, wherein the reduced number is determined based on a number of the at least some of the incoming code execution requests and the determined degree of periodicity;

cause at least one virtual machine instance to be added to the active pool before the time period, and cause a program code associated with the at least some of the incoming code execution requests to be loaded on the at least one virtual machine instance; and

in response to receiving a request associated with the at least some of the incoming code execution requests, cause the program code loaded on the at least one virtual machine to be executed.

2. The system of claim 1, wherein the virtual compute system is further configured to:

receive a first request to execute a first program code at a first time period;

receive a second request to execute a second program code at a second time period, wherein the second time period at least partially overlaps the first time period; and

schedule a first execution of the first program code and a second execution of the second program code such that the first and second executions do not overlap.

3. The system of claim 2, wherein the first time period comprises one of (i) a time by which the virtual compute system is requested to execute the first program code, (ii) a time after which the virtual compute system is requested to execute the first program code, or (iii) a temporal window within which the virtual compute system is requested to execute the first program code.

4. A system, comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices;

monitor incoming code execution requests to execute program code on the virtual compute system;

determine whether at least some of the incoming code execution requests exhibit periodicity, the at least some of the incoming code execution requests associated with one or more execution parameters;

in response to determining that the at least some of the incoming code execution requests exhibit periodicity, cause a reduced amount of idle compute capacity to be maintained on the virtual compute system, the reduced amount determined based on the periodicity; and

cause an additional virtual machine instance to be pre-initialized based on the one or more execution parameters before an additional periodic request exhibiting the same periodicity as the at least some of the incoming code execution requests is received by the virtual compute system.

5. The system of claim 4, wherein the plurality of virtual machine instances comprise an active pool of virtual machine instances, wherein the virtual compute system is configured to cause the reduced amount of idle compute capacity by removing from the active pool an instance that was previously used to execute a program code but is not currently executing a program code.

6. The system of claim 4, wherein the plurality of virtual machine instances comprise an active pool of virtual machine instances, and wherein the virtual compute system is configured to cause an additional virtual machine instance to be pre-initialized based on the one or more execution parameters by:

requesting the additional virtual machine instance from an instance provisioning service in networked communication with the virtual compute system;

causing the additional virtual machine instance to be added to the active pool; and

creating a container on the additional virtual machine and causing a program code associated with the additional periodic request to be loaded in the container.

7. The system of claim 4, wherein the plurality of virtual machine instances comprise a warming pool of virtual machine instances, and wherein the virtual compute system is configured to cause an additional virtual machine instance to be pre-initialized based on the one or more execution parameters by:

requesting the additional virtual machine instance from an instance provisioning service in networked communication with the virtual compute system;

causing the additional virtual machine instance to be added to the warming pool; and

creating a container on the additional virtual machine and causing a program code associated with the additional periodic request to be loaded in the container.

8. The system of claim 4, wherein the plurality of virtual machine instances comprise an active pool of virtual machine instances, and wherein the virtual compute system is configured to cause an additional virtual machine instance to be pre-initialized based on the one or more execution parameters by:

locating an unfilled virtual machine instance in the active pool, wherein the unfilled virtual machine instance is not fully utilized; and

creating a container on the unfilled virtual machine instance and causing a program code associated with the additional periodic request to be loaded in the container.

9. The system of claim 4, wherein the plurality of virtual machine instances comprise a warming pool of virtual machine instances having one or more software components loaded thereon and waiting to be assigned to a user, and wherein the virtual compute system is configured to cause a reduced amount of idle compute capacity to be maintained on the virtual compute system by refraining from adding additional virtual machine instances to the warming pool until a number of virtual machine instances in the warming pool reaches a number corresponding to the reduced amount.

10. The system of claim 4, wherein the virtual compute system is further configured to, in response to receiving the additional periodic request, cause a program code associated with the additional periodic request to be executed in a container created on the additional virtual machine instance, wherein the program code is loaded in the container before the additional periodic request is received by the virtual compute system.

11. The system of claim 4, wherein the virtual compute system is further configured to:

receive a first job request associated with a first program code, a first maximum duration for executing the first program code, and a first time frame for executing the first program code;

receive a second job request associated with a second program code and a second time frame for executing the second program code, the second time frame at least partially overlapping the first time frame; and

determine a first execution time at which the first program code is to be executed and a second execution time at which the second program code is to be executed such that the first execution time precedes the second execution time at least by the first maximum duration.

12. The system of claim 11, wherein the virtual compute system is further configured to provide, via a user interface, an option of selecting between a first degree of temporal flexibility associated with a first cost and a second degree of temporal flexibility that is greater than the first degree and associated with a second cost, wherein the first cost is greater than the second cost.

13. The system of claim 4, wherein the virtual compute system is configured to determine whether at least some of the incoming code execution requests exhibit periodicity by periodically analyzing log data generated based on the incoming code execution requests.

14. A computer-implemented method comprising:

as implemented by one or more computing devices configured with specific executable instructions,

maintaining a plurality of virtual machine instances on one or more physical computing devices;

monitoring incoming code execution requests to execute program code on the virtual compute system;

determining whether at least some of the incoming code execution requests exhibit periodicity, the at least some of the incoming code execution requests associated with one or more execution parameters;

in response to determining that the at least some of the incoming code execution requests exhibit periodicity, causing a reduced amount of unallocated compute capacity to be maintained on the virtual compute system, the reduced amount determined based on the periodicity;

determining that the at least some of the incoming code execution requests are expected to be received within a threshold amount of time; and

causing an additional virtual machine instance to be configured based on the one or more execution parameters before an additional periodic request exhibiting the same periodicity as the at least some of the incoming code execution requests is received by the virtual compute system.

15. The computer-implemented method of claim 14, wherein the plurality of virtual machine instances comprise an active pool of virtual machine instances, and wherein causing an additional virtual machine instance to be configured based on the one or more execution parameters comprises:

requesting the additional virtual machine instance from an instance provisioning service in networked communication with the virtual compute system;

causing the additional virtual machine instance to be added to the active pool; and

creating a container on the additional virtual machine and causing a program code associated with the at least some of the incoming code execution requests to be loaded in the container.

16. The computer-implemented method of claim 14, wherein the plurality of virtual machine instances comprise a warming pool of virtual machine instances, and wherein causing an additional virtual machine instance to be configured based on the one or more execution parameters comprises:

requesting the additional virtual machine instance from an instance provisioning service in networked communication with the virtual compute system;

causing the additional virtual machine instance to be added to the warming pool; and

creating a container on the additional virtual machine and causing a program code associated with the additional periodic request to be loaded in the container.

17. The computer-implemented method of claim 14, further comprising:

receiving a request to execute a first job associated with a first set of execution requirements;

receiving a request to execute a second job associated with a second set of execution requirements;

determining one or more scheduling criteria based on the first and second sets of execution requirements; and

determining a first execution time for executing the first job and a second execution time for executing the second job such that the one or more scheduling criteria are satisfied.

18. Non-transitory physical computer storage comprising instructions that, when executed by one or more computing devices, configure the one or more computing devices to:

maintain a plurality of virtual machine instances on one or more physical computing devices;

monitor incoming code execution requests to execute program code on the virtual compute system;

determine whether at least some of the incoming code execution requests exhibit periodicity, the at least some of the incoming code execution requests associated with one or more execution parameters;

in response to determining that the at least some of the incoming code execution requests exhibit periodicity, cause a reduced amount of unallocated compute capacity to be maintained on the virtual compute system, the reduced amount determined based on the periodicity;

determine that the at least some of the incoming code execution requests are expected to be received within a threshold amount of time; and

cause an additional virtual machine instance to be configured based on the one or more execution parameters before an additional periodic request exhibiting the same periodicity as the at least some of the incoming code execution requests is received by the virtual compute system.

19. The non-transitory physical computer storage of claim 18, wherein the plurality of virtual machine instances comprise an active pool of virtual machine instances, and wherein causing an additional virtual machine instance to be configured based on the one or more execution parameters comprises:

requesting the additional virtual machine instance from an instance provisioning service in networked communication with the virtual compute system;

causing the additional virtual machine instance to be added to the active pool; and

creating a container on the additional virtual machine and causing a program code associated with an additional periodic request exhibiting the same periodicity as the at least some of the incoming code execution requests to be loaded in the container.

20. The non-transitory physical computer storage of claim 18, wherein the plurality of virtual machine instances comprise an active pool of virtual machine instances, and wherein causing an additional virtual machine instance to be configured based on the one or more execution parameters comprises:

locating an unfilled virtual machine instance in the active pool, wherein the unfilled virtual machine instance is not fully utilized; and

creating a container on the unfilled virtual machine instance and causing a program code associated with the additional periodic request to be loaded in the container.

21. The non-transitory physical computer storage of claim 18, wherein the instructions further configure the one or more computing devices to:

receive a request to execute a first job associated with a first set of execution requirements;

receive a request to execute a second job associated with a second set of execution requirements;

determine one or more scheduling criteria based on the first and second sets of execution requirements; and

determine a first execution time for executing the first job and a second execution time for executing the second job such that the one or more scheduling criteria are satisfied.