Pre-triggers for code execution environments

• US 10,013,267 B1
• Filed: 12/16/2015
• Issued: 07/03/2018
• Est. Priority Date: 12/16/2015
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen, Brooker, Marc John
• Primary Examiner(s): Kim, Dong
• Application Number:
  US14/971,934
• Time in Patent Office: 930 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5005 to service a request

Abstract

Systems and methods are described for managing initialization of virtual machine instances within an on-demand code execution environment or other distributed code execution environment. Such environments utilize pre-initialized virtual machine instances to enable execution of user-specified code in a rapid manner, without delays typically caused by initialization of the virtual machine instances. However, because the number of pre-initialized virtual machine instances maintained at an on-demand code execution environment is typically limited, insufficient number of pre-initialized virtual machine instances may be available at the on-demand code execution environment during times of heavy use. Embodiments described herein utilize pre-trigger notifications to indicate to the on-demand code execution environment that subsequent requests to execute user-specified code are likely to occur. The on-demand code execution environment may therefore pre-initialize additional virtual machine instances in preparation for the subsequent requests, reducing delay that would be required to initialize the instances after obtaining to the requests.

Claims

1. A system for providing low-latency computational capacity in an on-demand code execution environment, the system comprising:

a data store including historical data regarding use of the on-demand code execution environment, the historical data correlating pre-trigger notifications previously obtained at the on-demand code execution environment with one or more requests to execute first user-specified code obtained at the on-demand code execution environment subsequent to individual pre-trigger notifications, wherein the pre-trigger notifications are obtained from one or more computing systems associated with the user-specified code and distinct from the on-demand code execution environment, and wherein the pre-trigger notifications are requests to execute second user-specified code that are distinct from the requests to execute the first user-specified code that are obtained subsequent to the individual pre-trigger notifications; and

one or more computing devices in communication with the data store and executing the computer-executable instructions, wherein execution of the computer-executable instructions causes the one or more computing devices to;

maintain a set of virtual machine instances that are pre-initialized with an operating environment and made available for execution of the first user-specified code;

obtain, from the one or more computing systems, an additional pre-trigger notification requesting execution of the second user-specified code;

determine, based at least in part on comparing the additional pre-trigger notification to the historical data correlating pre-trigger notifications previously obtained at the on-demand code execution environment, a number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the additional pre-trigger notification;

based on the number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the additional pre-trigger notification, increase a number of virtual machine instances within the set of virtual machine instances at least partly by initializing at least one new virtual machine instance with an operating environment and adding the at least one new virtual machine instance to the set of virtual machine instances;

obtain a request to execute the first user-specified code, wherein the request is associated with the pre-trigger notification; and

execute the first user-specified code on the previously initialized operating environment of the at least one new virtual machine instance.

2. The system of claim 1, wherein execution of the computer-executable instructions causes the one or more computing devices to generate the historical data based at least in part on:

for individual pre-trigger notifications of a plurality of pre-trigger notifications obtained at the on-demand code execution environment, determining a set of requests to execute the user-specified code occurring subsequent to the individual pre-trigger notifications;

generate statistical information indicating a likelihood that an individual pre-trigger notification will result in a subsequent number of requests to execute the user-specified code;

store the statistical information within the data store as the historical data.

3. The system of claim 1, wherein execution of the computer-executable instructions causes the one or more computing devices to generate the historical data based at least in part on:

identifying a set of pre-trigger notifications that are associated with one another;

determining a set of requests to execute the first user-specified code occurring subsequent to the set of pre-trigger notifications;

generate statistical information indicating a likelihood that the set of pre-trigger notifications will result in a subsequent number of requests to execute the first user-specified code;

store the statistical information within the data store as the historical data.

4. The system of claim 1, wherein the request to execute the first user-specified code comprises at least one of an application programming interface (API) call, a hypertext transport protocol (HTTP) transmission, or a request generated by the on-demand code execution environment in response to data obtained from an external service.

5. The system of claim 4, wherein execution of the computer-executable instructions further causes the one or more computing devices to increase a rate at which the external service is polled for the data.

6. The system of claim 1, wherein the pre-trigger notification indicates an anticipated number of requests to execute the user-specified code, and wherein determining the number of requests to execute the user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification includes modifying the anticipated number of requests based at least in part on the historical data.

7. The system of claim 1, wherein each pre-trigger notification corresponds to a type of event, and wherein the data store further includes historical data for one or more additional pre-trigger notifications corresponding to different types of events.

8. A computer-implemented method of providing low-latency computational capacity in an on-demand code execution environment, the computer-implemented method comprising:

maintaining a set of execution environments that are pre-initialized with an operating environment and made available for execution of first user-specified code;

obtaining a pre-trigger notification indicating that a likelihood that one or more requests to execute first user-specified code will be obtained at the on-demand code execution environment subsequent to the pre-trigger notification, wherein the pre-trigger notification is obtained from one or more computing systems associated with the first user-specified code and distinct from the on-demand code execution environment, and wherein the pre-trigger notification is a request to execute second user-specified code and is distinct from the one or more requests to execute the first user-specified code;

determining, based at least in part on the pre-trigger notification representing the request to execute the second user-specific code, a number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification;

based at least partly on the number of requests that are predicted to occur, managing a number of execution environments within the set of execution environments;

obtaining a request to execute the first user-specified code, wherein the request is associated with the pre-trigger notification; and

executing the first user-specified code on at least one execution environment of the set of execution environments.

9. The computer-implemented method of claim 8, wherein the set of execution environments includes at least one of a set of virtual machine instances or a set of software containers.

10. The computer-implemented method of claim 8, wherein managing the number of execution environments within the set of execution environments includes at least one of increasing the number of execution environments within the set of execution environments, declining to de-initialize at least one execution environment within the set of execution environments that would otherwise be designated for de-initialization, or modifying a set of policies that determine when execution environments are de-initialized.

11. The computer-implemented method of claim 8 further comprising generating statistical information regarding previous pre-trigger notifications received at the on-demand code execution environment and requests to execute the first user-specified code that are correlated to those previous pre-trigger notifications, wherein the number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification is determined based at least in part on the statistical information.

12. The computer-implemented method of claim 11, wherein the statistical information includes a distribution modeling a number and timing of requests to execute the first user-specified code that are expected to occur subsequent to an individual pre-trigger notification.

13. The computer-implemented method of claim 8 further comprising determining a number of virtual machine instances capable of servicing the requests predicted to occur subsequent to obtaining the pre-trigger notification.

14. The computer-implemented method of claim 13, wherein the number of virtual machine instances capable of servicing the requests predicted to occur subsequent to obtaining the pre-trigger notification is determined based at least in part on the first user-specified code.

15. The computer-implemented method of claim 13, wherein increasing the number of virtual machine instances within the set of virtual machine instances comprises increasing the number of virtual machine instances within the set of virtual machine instances by at least the number of virtual machine instances capable of servicing the requests predicted to occur subsequent to obtaining the pre-trigger notification.

16. The computer-implemented method of claim 8 further comprising determining an expected timing of the requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification.

17. The computer-implemented method of claim 16, wherein increasing a number of virtual machine instances within the set of virtual machine instances comprises increasing a number of virtual machine instances within the set of virtual machine instances in accordance with the expected timing of the requests.

18. The computer-implemented method of claim 16 further comprising, based at least partly on the number of requests that are predicted to occur, managing computing resources available to the set of execution environments.

19. Non-transitory computer-readable storage media comprising computer-executable instructions executable by a computing system associated with an on-demand code execution environment to cause the computing system to:

obtain a pre-trigger notification indicating that a likelihood that one or more requests to execute first user-specified code will be obtained at the on-demand code execution environment subsequent to the pre-trigger notification, wherein the pre-trigger notification is obtained from one or more computing systems associated with the user-specified code and distinct from the on-demand code execution environment, and wherein the pre-trigger notification is a request to execute second user-specified code and is distinct from the one or more requests to execute the first user-specified code;

determine, based at least in part on the pre-trigger notification, a number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification;

based at least partly on the number of requests that are predicted to occur, managing a number of execution environments on the on-demand code execution environment that are available to execute the first user-specified code;

obtain at least one request to execute the first user-specified code, wherein the request is associated with the pre-trigger notification; and

transmit instructions to the set of execution environments to execute the first user-specified code on the previously initialized operating environment.

20. The non-transitory computer-readable storage media of claim 19, wherein the computer-executable instructions cause the computing system to determine the number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification at least partly by extracting, from the pre-trigger notification, an anticipated number of requests indicated within the pre-trigger notification.

21. The non-transitory computer-readable storage media of claim 19, wherein the computer-executable instructions further cause the computing system to generate statistical information regarding previous pre-trigger notifications received at the on-demand code execution environment and requests to execute the first user-specified code that are correlated to those previous pre-trigger notifications, wherein the number of requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification is determined based at least in part on the statistical information.

22. The non-transitory computer-readable storage media of claim 19, wherein the computer-executable instructions further cause the computing system determine an expected timing of the requests to execute the first user-specified code that are predicted to occur subsequent to obtaining the pre-trigger notification.

23. The non-transitory computer-readable storage media of claim 19, wherein the computer-executable instructions further cause the computing system to determine a number of virtual machine instances capable of servicing the requests predicted to occur subsequent to obtaining the pre-trigger notification.

24. The non-transitory computer-readable storage media of claim 23, wherein the at least one new virtual machine instance includes at least the number of virtual machine instances capable of servicing the requests predicted to occur subsequent to obtaining the pre-trigger notification is determined based at least in part on the first user-specified code.