Predictive management of on-demand code execution

• US 9,811,363 B1
• Filed: 12/16/2015
• Issued: 11/07/2017
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen
• Primary Examiner(s): Nguyen, Van
• Application Number:
  US14/971,876
• Time in Patent Office: 692 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/44521 Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5033 considering data affinity

Abstract

Systems and methods are described for predictively managing calls to tasks on an on-demand code execution environment. Specifically, a task profile can be utilized to predict that a call to a first task will be followed by a call to a second task. Thereafter, a virtual machine can be loaded with executable code corresponding the second task, even before the call to the second task is received. If the call to the second task is received, the task can then be executed by utilizing the previously loaded executable code, thus increasing the speed at which the second task can be executed on the on-demand code execution environment.

Claims

1. A computer-implemented method for predictively managing code execution in an on-demand code execution environment, the on-demand code execution environment comprising at least one virtual machine instance executing an initialized operating system and awaiting requests to execute user-specified code within that operating system, the method comprising:

as implemented by one or more computing devices configured with specific computer-executable instructions,

obtaining a task profile for a first task associated with the on-demand code execution environment, the first task corresponding to a first set of user-defined code executable by the virtual machine instance, wherein the task profile indicates a likelihood that a request to execute the first task on the on-demand code execution environment will be followed by a request to execute a second task corresponding to a second set of user-defined code executable by the virtual machine instance;

obtaining the request to execute the first task on the on-demand code execution environment;

instructing the virtual machine instance to execute the first set of user-defined code;

determining that the likelihood indicated by the task profile satisfies a threshold value;

prior to receiving the request to execute the second task, loading the virtual machine instance with the second set of user-defined code;

receiving the request to execute the second task; and

instructing the virtual machine instance to execute the second set of user-defined code previously loaded on the virtual machine instance.

2. The computer-implemented method of claim 1, wherein the request to execute the first task corresponds to at least one of an application programming interface (API) call, a hypertext transport protocol (HTTP) request, a request generated by execution of a task on the on-demand code execution environment, or a request generated by the on-demand code execution environment based on data obtained from an auxiliary service.

3. The computer-implemented method of claim 1, wherein loading the virtual machine instance with the second set of user-defined code comprises:

combining first set of user-defined code and the second set of user-defined code into a combined set of code;

modifying the combined set of code to replace a portion of the combined set of code that generates the request to execute the second task with a reference to a portion of the combined set of code that corresponds to the second set of user-defined code; and

loading the combined set of code on the virtual machine instance.

4. The computer-implemented method of claim 1, wherein the task profile includes a directed graph identifying a statistical relationship between an individual execution of the first task and calls caused by the individual execution.

5. The computer-implemented method of claim 4, wherein nodes of the directed graph identify at least one of a task on the on-demand code execution environment or an auxiliary service, and wherein edges of the directed graph indicate calls between the nodes of the directed graph.

6. A system for predictively managing code execution in an on-demand code execution environment, the on-demand code execution environment comprising at least one virtual machine instance executing an initialized operating system and awaiting requests to execute user-specified code within that operating system, the system comprising:

a non-transitory data store including a task profile for a first task associated with the on-demand code execution environment, the first task corresponding to a first set of user-defined code executable by the virtual machine instance, wherein the task profile indicates a likelihood that a request to execute the first task on the on-demand code execution environment will be followed by a request to execute a second task corresponding to a second set of user-defined code executable by the virtual machine instance; and

a computing device comprising a processor in communication with the non-transitory data store and configured with specific computer-executable instructions to;

obtain the request to execute the first task on the on-demand code execution environment;

determine that the likelihood indicated by the task profile satisfies a threshold value;

prior to receiving the request to execute the second task, load a virtual machine instance with the second set of user-defined code;

obtain the request to execute the second task; and

execute, on the virtual machine instance, the second set of user-defined code previously loaded on the virtual machine instance.

7. The system of claim 6, wherein the specific computer-executable instructions further configure the computing device to execute the first set of user-defined code on the virtual machine instance.

8. The system of claim 6, wherein the specific computer-executable instructions further configure the computing device to execute the first set of user-defined code on a second virtual machine instance.

9. The system of claim 6, wherein the request to execute the second task is generated by the virtual machine instance during execution of the first set of user-defined code and is intended for a frontend component of the on-demand code execution environment, and wherein obtaining the request to execute the second task comprises intercepting, at the virtual machine instance, the request to execute the second task.

10. The system of claim 6, wherein the specific computer-executable instructions further configure the computing device to verify that the second task satisfies eligibility criteria for preemptive loading of the second set of user-defined code.

11. The system of claim 10, wherein the eligibility criteria includes at least one of synchronicity of the second task, execution time of the second task, or operating environment required by the second task.

12. The system of claim 6, wherein the specific computer-executable instructions configure the computing device to load the virtual machine instance with the second set of user-defined code at least in part by:

combining first set of user-defined code and the second set of user-defined code into a combined set of code;

modifying the combined set of code to replace a portion of the combined set of code that generates the request to execute the second task with a reference to a portion of the combined set of code that corresponds to the second set of user-defined code; and

loading the combined set of code on the virtual machine instance.

13. The system of claim 6, wherein the task profile further indicates a likelihood that execution of the first task on the on-demand code execution environment will cause transmission of information to an auxiliary service, and wherein the specific computer-executable instructions further configure the computing device to:

determine that the likelihood that execution of the first task on the on-demand code execution environment will cause transmission of information to the auxiliary service satisfies a second threshold value; and

select a virtual machine instance to execute the first set of user-defined code based at least in part on a quality of communication channel between the selected virtual machine instance and the auxiliary service.

14. Non-transitory, computer-readable storage media comprising computer-executable instructions for predictively managing code execution in an on-demand code execution environment, wherein the computer-executable instructions, when executed by a computer system, cause the computer system to:

obtain a request to execute a first task on the on-demand code execution environment;

determine that execution of the first task on the on-demand code execution environment is likely to result in a request to execute a second task on the on-demand code execution environment, wherein the first task corresponds to a first set of user-defined code executable by the on-demand code execution environment, wherein the second task corresponds to a second set of user-defined code executable by the on-demand code execution environment, and wherein said determination is based at least in part on a task profile for the first task that indicates a likelihood that the request to execute the first task will be followed by the request to execute the second task; and

prior to receiving the request to execute the second task, load a virtual machine instance with the second set of user-defined code;

obtain the request to execute the second task; and

execute, on the virtual machine instance, the second set of user-defined code previously loaded on the virtual machine instance.

15. The non-transitory, computer-readable storage media of claim 14, wherein the computer-executable instructions further cause the computer system to obtain the task profile from a task profiler of the on-demand code execution environment.

16. The non-transitory, computer-readable storage media of claim 14, wherein the specific computer-executable instructions further cause the computer system to execute the first set of user-defined code on the virtual machine instance.

17. The non-transitory, computer-readable storage media of claim 14, wherein the specific computer-executable instructions further cause the computer system to execute the first set of user-defined code on a second virtual machine instance.

18. The non-transitory, computer-readable storage media of claim 14, wherein the specific computer-executable instructions further cause the computer system to load the virtual machine instance with the second set of user-defined code at least in part by:

combining first set of user-defined code and the second set of user-defined code into a combined set of code;

modifying the combined set of code to replace a portion of the combined set of code that generates the request to execute the second task with a reference to a portion of the combined set of code that corresponds to the second set of user-defined code; and

loading the combined set of code on the virtual machine instance.

19. The non-transitory, computer-readable storage media of claim 14, wherein obtaining the request to execute the second task comprises executing a portion of the first set of user-defined code that references the second set of user-defined code.

20. The non-transitory, computer-readable storage media of claim 14, wherein the task profile further indicates a likelihood that execution of the first task on the on-demand code execution environment will cause transmission of information to an auxiliary service, and wherein the specific computer-executable instructions further cause the computer system to:

determine that the likelihood that execution of the first task on the on-demand code execution environment will cause transmission of information to the auxiliary service satisfies a second threshold value; and

select a virtual machine instance to execute the first set of user-defined code based at least in part on a quality of communication channel between the selected virtual machine instance and the auxiliary service.