Predictive management of on-demand code execution

• US 9,830,175 B1
• Filed: 12/16/2015
• Issued: 11/28/2017
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen
• Primary Examiner(s): Nguyen, Van
• Application Number:
  US14/971,977
• Time in Patent Office: 713 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/44521 Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

Systems and methods are described for predictively managing calls to tasks on an on-demand code execution environment. Specifically, a task profile can be utilized to predict that a call to a task will be followed by transmission of information to an auxiliary service. Thereafter, the on-demand code execution environment can select a virtual machine to execute the task based, for example, on the speed or reliability of a communication channel between the selected virtual machine and the auxiliary service. If execution of the task does cause transmission of information to the auxiliary service, the information can be transmitted via the communication channel, thereby increasing the speed or reliability of the transmission.

Claims

1. A computer-implemented method for predictively managing code execution in an on-demand code execution environment, the on-demand code execution environment comprising a plurality of virtual machine instances, individual virtual machine instances of the plurality of virtual machine instances executing an initialized operating system and awaiting requests to execute user-specified code within that operating system, the method comprising:

as implemented by one or more computing devices configured with specific computer-executable instructions,

obtaining a task profile for a task associated with the on-demand code execution environment, the task corresponding to a set of user-defined code executable by at least one of the plurality of virtual machine instances, wherein the task profile indicates a likelihood that an execution of the task on the on-demand code execution environment will cause transmission of information to an auxiliary service external to the on-demand code execution environment;

obtaining the request to execute the task on the on-demand code execution environment;

determining that the likelihood indicated by the task profile satisfies a threshold value;

selecting a virtual machine instance from the plurality of virtual machine instances on which to execute the set of user-defined code based at least in part on a quality of communication channel between the virtual machine instance and the auxiliary service; and

instructing the virtual machine instance to execute the first set of user-defined code, wherein execution of the first set of user-defined code on the virtual machine instance results in the transmission of information to the auxiliary service via the communication channel.

2. The computer-implemented method of claim 1, wherein the request to execute the first task corresponds to at least one of an application programming interface (API) call, a hypertext transport protocol (HTTP) request, a request generated by execution of a task on the on-demand code execution environment, or a request generated by the on-demand code execution environment based on data obtained from an auxiliary service.

3. The computer-implemented method of claim 1, wherein the quality of communication channel is determined based at least in part on at least one of a network distance, a bandwidth, or a reliability of the communication channel, a protocol utilized by the virtual machine instance to communicate with the auxiliary service, or an existence of a pre-established connection between the virtual machine instance and the auxiliary service.

4. The computer-implemented method of claim 3, wherein the network distance reflects at least one of a latency of the communication channel or a number of intermediary devices on the communication channel.

5. The computer-implemented method of claim 1, wherein the task profile includes a directed graph identifying a statistical relationship between an individual execution of the first task and calls caused by the individual execution.

6. The computer-implemented method of claim 5, wherein nodes of the directed graph identify at least one of a task on the on-demand code execution environment or an auxiliary service, and wherein edges of the directed graph indicate calls between the nodes of the directed graph.

7. A system for predictively managing code execution in an on-demand code execution environment, the on-demand code execution environment comprising a plurality of virtual machine instances, individual virtual machine instances of the plurality of virtual machine instances executing an initialized operating system and awaiting requests to execute user-specified code within that operating system, the system comprising:

a non-transitory data store including a task profile for a task associated with the on-demand code execution environment, the task corresponding to a set of user-defined code executable by at least one of the plurality of virtual machine instances, wherein the task profile indicates a likelihood that an execution of the task on the on-demand code execution environment will cause transmission of information to an auxiliary service external to the on-demand code execution environment; and

a computing device comprising a processor in communication with the non-transitory data store and configured with specific computer-executable instructions to;

obtain the request to execute the task on the on-demand code execution environment;

determine that the likelihood indicated by the task profile satisfies a threshold value;

select a virtual machine instance from the plurality of virtual machine instances on which to execute the set of user-defined code based at least in part on a communication channel between the virtual machine instance and the auxiliary service;

execute the set of user-defined code on the virtual machine instance; and

during execution of the first set of user-defined code, transmit information to the auxiliary service via the communication channel.

8. The system of claim 7, wherein the specific computer-executable instructions further configure the computing device, prior to executing a portion of the set of user-defined code that causes transmission of information to the auxiliary service, to instruct the virtual machine instance to establish a communication session with the auxiliary service.

9. The system of claim 7, wherein the communication channel includes a private network.

10. The system of claim 7, wherein the specific computer-executable instructions configure the computing device to select the virtual machine instance on which to execute the set of user-defined code based at least in part on the communication channel at least partly by verifying that a network quality metric of the communication channel satisfies a second threshold value.

11. The system of claim 10, wherein the network quality metric measures at least one of a network distance, a bandwidth, or a reliability of the communication channel, a protocol utilized by the virtual machine instance to communicate with the auxiliary service via the communication channel, or availability of a pre-established connection on the communication channel between the virtual machine instance and the auxiliary service.

12. The system of claim 7, wherein the task profile further indicates a likelihood that execution of the task on the on-demand code execution environment will be followed by a request to execute a second task corresponding to a second set of user-defined code, and wherein the specific computer-executable instructions further configure the computing device to:

determine that the likelihood that execution of the task on the on-demand code execution environment will be followed by the request to execute the second task satisfies a second threshold value; and

prior to receiving the request to execute the second task, load a virtual machine instance from the plurality of virtual machine instances with the second set of user-defined code.

13. The system of claim 12, wherein the virtual machine instance selected to execute the set of user-defined code is the same as the virtual machine instance on which the second set of user-defined code is loaded.

14. The system of claim 12, wherein the virtual machine instance selected to execute the set of user-defined code is different than the virtual machine instance on which the second set of user-defined code is loaded.

15. Non-transitory, computer-readable storage media comprising computer-executable instructions for predictively managing code execution in an on-demand code execution environment, wherein the computer-executable instructions, when executed by a computer system, cause the computer system to:

obtain a request to execute a task on the on-demand code execution environment;

determine that execution of the task on the on-demand code execution environment is likely to result in transmission of information to an auxiliary service external to the on-demand code execution environment, wherein the task corresponds to a set of user-defined code executable by the on-demand code execution environment, and wherein said determination is based at least in part on a task profile for the task that indicates a likelihood that the request to execute the task will result in transmission of information to the auxiliary service;

select a virtual machine instance, from a plurality of virtual machine instances associated with the on-demand code execution environment, on which to execute the set of user-defined code based at least in part on a communication channel between the virtual machine instance and the auxiliary service; and

during execution of the set of user-defined code, transmit the information to the auxiliary service via the communication channel.

16. The non-transitory, computer-readable storage media of claim 15, wherein the computer-executable instructions further cause the computer system to obtain the task profile from a task profiler of the on-demand code execution environment.

17. The non-transitory, computer-readable storage media of claim 15, wherein the computer-executable instructions further cause the computer system, prior to executing a portion of the set of user-defined code that causes transmission of information to the auxiliary service, to instruct the virtual machine instance to establish a communication session with the auxiliary service.

18. The non-transitory, computer-readable storage media of claim 15, wherein the computer-executable instructions cause the computer system to select the virtual machine instance on which to execute the set of user-defined code based at least in part on the communication channel at least partly by verifying that a network quality metric of the communication channel satisfies a second threshold value.

19. The non-transitory, computer-readable storage media of claim 15, wherein the task profile further indicates a likelihood that execution of the task on the on-demand code execution environment will be followed by a request to execute a second task corresponding to a second set of user-defined code, and wherein the specific computer-executable instructions further configure the computing device to:

determine that the likelihood that execution of the task on the on-demand code execution environment will be followed by the request to execute the second task satisfies a second threshold value; and

prior to receiving the request to execute the second task, load a virtual machine instance from the plurality of virtual machine instances with the second set of user-defined code.

20. The non-transitory, computer-readable storage media of claim 19, wherein the virtual machine instance selected to execute the set of user-defined code is the same as the virtual machine instance on which the second set of user-defined code is loaded.