Pre-triggers for code execution environments

• US 10,437,629 B2
• Filed: 07/02/2018
• Issued: 10/08/2019
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen, Brooker, Marc John
• Primary Examiner(s): Kim, Dong U
• Application Number:
  US16/025,694
• Publication Number:
  US 20190073234A1
• Time in Patent Office: 463 Days
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5005 to service a request

Abstract

Systems and methods are described for managing initialization of virtual machine instances within an on-demand code execution environment or other distributed code execution environment. Such environments utilize pre-initialized virtual machine instances to enable execution of user-specified code in a rapid manner, without delays typically caused by initialization of the virtual machine instances. However, because the number of pre-initialized virtual machine instances maintained at an on-demand code execution environment is typically limited, insufficient number of pre-initialized virtual machine instances may be available at the on-demand code execution environment during times of heavy use. Embodiments described herein utilize pre-trigger notifications to indicate to the on-demand code execution environment that subsequent requests to execute user-specified code are likely to occur. The on-demand code execution environment may therefore pre-initialize additional virtual machine instances in preparation for the subsequent requests, reducing delay that would be required to initialize the instances after obtaining to the requests.

Claims

1. A computer-implemented method comprising:

obtaining a request to execute first user-specified code at an on-demand code execution environment, wherein the request to execute first user-specified code is obtained from one or more computing systems distinct from the on-demand code execution environment;

obtaining historical data correlating prior requests to execute the first user-specified code with subsequent requests to execute second user-specified code that is distinct from the first user-specified code;

determining, based at least in part on the historical data, that a request to execute the second user-specified code is predicted to occur subsequent to obtaining the request to execute the first user-specified code;

based at least partly on determining that the request to execute the second user-specified code is predicted to occur, and prior to receiving the request to execute the second user-specified code, initializing at least one virtual machine instance with an operating environment configured to enable execution of the second user-specified code;

obtaining the request to execute the second user-specified code; and

executing the second user-specified code within the operating environment.

2. The computer-implemented method of claim 1, wherein determining that the request to execute the second user-specified code is predicted to occur comprises determining a statistical likelihood that the request to execute the second user-specified code is predicted to occur and that the statistical likelihood satisfies a threshold value.

3. The computer-implemented method of claim 1 further comprising determining, based at least in part on the historical data, a point in time at which the request to execute the second user-specified code is predicted to occur, wherein initializing at least one virtual machine instance comprises initializing the at least one virtual machine instance at a point in time selected based at least partly on the point in time at which the request to execute the second user-specified code is predicted to occur.

4. The computer-implemented method of claim 1 further comprising generating the historical data, wherein generating the historical data comprises:

for individual prior requests to execute the first user-specified code, determining whether a subsequent request to execute the second user-specified code occurred;

generating statistical information indicating a likelihood that an individual request to execute the first user-specified code will result in a subsequent request to execute the second user-specified code; and

storing the statistical information as the historical data.

5. The computer-implemented method of claim 1, wherein determining that a request to execute the second user-specified code is predicted to occur comprises determining that a plurality of requests to execute the second user-specified code are predicted to occur, and wherein initializing at least one virtual machine instance with the operating environment configured to enable execution of the second user-specified code comprises initializing a plurality of virtual machine instances with the operating environment.

6. The computer-implemented method of claim 1, wherein the request to execute the first user-specified code comprises at least one of an application programming interface (API) call, a hypertext transport protocol (HTTP) transmission, or a request generated by the on-demand code execution environment in response to data obtained from an external service.

7. A system comprising:

a processor; and

a data store including computer-executable instructions executable by the processor that cause the system to;

obtain a request to execute first user-specified code at an on-demand code execution environment, wherein the request to execute the first user-specified code is obtained from one or more computing systems distinct from the on-demand code execution environment;

obtain historical data correlating prior requests to execute the first user-specified code with subsequent requests to execute second user-specified code that is distinct from the first user-specified code;

determine, based at least in part on the historical data, that a request to execute the second user-specified code is predicted to occur subsequent to obtaining the request to execute the first user-specified code;

based at least partly on determining that the request to execute the second user-specified code is predicted to occur, and prior to receiving the request to execute the second user-specified code, configure at least one virtual machine instance with an operating environment configured to enable execution of the second user-specified code;

obtain the request to execute the second user-specified code; and

execute the second user-specified code within the operating environment.

8. The system of claim 7, wherein, to determine that the request to execute the second user-specified code is predicted to occur, the computer-executable instructions are further executable by the processor to cause the system to determine a statistical likelihood that the request to execute the second user-specified code is predicted to occur and that the statistical likelihood satisfies a threshold value.

9. The system of claim 8, wherein the historical data comprises a distribution modeling a frequency of requests to execute the second user-specified code over time subsequent to requests to execute the first user-specified code and wherein, to determine the statistical likelihood that the request to execute the second user-specified code is predicted to occur, the computer-executable instructions are further executable by the processor to reference the distribution.

10. The system of claim 7, wherein the computer-executable instructions are further executable by the processor to cause the system to determine, based at least in part on the historical data, a point in time at which the request to execute the second user-specified code is predicted to occur, and wherein, to configure the at least one virtual machine instance, the computer-executable instructions are further executable by the processor to cause the system to configure the at least one virtual machine instance at a point in time selected based at least partly on the point in time at which the request to execute the second user-specified code is predicted to occur.

11. The system of claim 7, wherein the computer-executable instructions are further executable by the processor to cause the system to:

for individual prior requests to execute the first user-specified code, determine whether a subsequent request to execute the second user-specified code occurred;

generate statistical information indicating a likelihood that an individual request to execute the first user-specified code will result in a subsequent request to execute the second user-specified code; and

store the statistical information as the historical data.

12. The system of claim 7, wherein, to determine that the request to execute the second user-specified code is predicted to occur, the computer-executable instructions are further executable by the processor to determine that a plurality of requests to execute the second user-specified code are predicted to occur, and wherein, to configure the at least one virtual machine instance with the operating environment configured to enable execution of the second user-specified code, the computer-executable instructions are further executable by the processor to configure a plurality of virtual machine instances with the operating environment.

13. The system of claim 7, wherein, to configure the at least one virtual machine instance with the operating environment, the computer-executable instructions are further executable by the processor to at least one of initialize the at least one virtual machine instance with the operating environment or prevent de-initialization of the at least one virtual machine instance with the operating environment.

14. Non-transitory computer readable media comprising computer-executable instructions that when executed by a computing system cause the computing system to:

obtain a request to execute first user-specified code at an on-demand code execution environment, wherein the request to execute first user-specified code is obtained from one or more computing systems distinct from the on-demand code execution environment;

obtain historical data correlating prior requests to execute the first user-specified code with subsequent requests to execute second user-specified code that is distinct from the first user-specified code;

determine, based at least in part on the historical data, that a request to execute the second user-specified code is predicted to occur subsequent to obtaining the request to execute the first user-specified code;

based at least partly on determining that the request to execute the second user-specified code is predicted to occur, and prior to receiving the request to execute the second user-specified code, configure at least one virtual machine instance with an operating environment configured to enable execution of the second user-specified code;

obtain the request to execute the second user-specified code; and

execute the second user-specified code within the operating environment.

15. The non-transitory computer-readable medium of claim 14, wherein, to determine that the request to execute the second user-specified code is predicted to occur, the computer-executable instructions are further executable to cause the computing system to determine a statistical likelihood that the request to execute the second user-specified code is predicted to occur and that the statistical likelihood satisfies a threshold value.

16. The non-transitory computer-readable medium of claim 14, wherein the computer-executable instructions are further executable to cause the computing system to determine, based at least in part on the historical data, a point in time at which the request to execute the second user-specified code is predicted to occur, and wherein, to initialize the at least one virtual machine instance, the computer-executable instructions are further executable to cause the computing system to initialize the at least one virtual machine instance at a point in time selected based at least partly on the point in time at which the request to execute the second user-specified code is predicted to occur.

17. The non-transitory computer-readable medium of claim 14, wherein the computer-executable instructions are further executable to cause the computing system to:

for individual prior requests to execute the first user-specified code, determine whether a subsequent request to execute the second user-specified code occurred;

generate statistical information indicating a likelihood that an individual request to execute the first user-specified code will result in a subsequent request to execute the second user-specified code; and

store the statistical information as the historical data.

18. The non-transitory computer-readable medium of claim 17, wherein the statistical information comprises a distribution modeling a frequency of subsequent requests to execute the second user-specified code over time.

19. The non-transitory computer-readable medium of claim 14, wherein, to determine that the request to execute the second user-specified code is predicted to occur, the computer-executable instructions are further executable to cause the computing system to determine that a plurality of requests to execute the second user-specified code are predicted to occur, and wherein, to initialize the at least one virtual machine instance with the operating environment configured to enable execution of the second user-specified code, the computer-executable instructions are further executable by the computing system to initialize a plurality of virtual machine instances with the operating environment.

20. The non-transitory computer-readable medium of claim 14, wherein, to configure the at least one virtual machine instance with the operating environment, the computer-executable instructions are further executable to cause the computing system to at least one of initialize the at least one virtual machine instance with the operating environment or prevent de-initialization of the at least one virtual machine instance with the operating environment.