Asynchronous task management in an on-demand network code execution environment

• US 9,952,896 B2
• Filed: 06/28/2016
• Issued: 04/24/2018
• Est. Priority Date: 06/28/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen, Brooker, Marc John, Nair, Ajay
• Primary Examiner(s): NGUYEN, VAN H
• Application Number:
  US15/195,897
• Publication Number:
  US 20170371706A1
• Time in Patent Office: 665 Days
• Field of Search: None
• CPC Class Codes: G06F 9/485 Task life-cycle, e.g. stopping, restarting, resuming execution G06F9/4881 takes precedence
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores

Abstract

Systems and methods are described for managing asynchronous code executions in an on-demand code execution system or other distributed code execution environment, in which multiple execution environments, such as virtual machine instances, can be used to enable rapid execution of user-submitted code. When asynchronous executions occur, one execution may become blocked while waiting for completion of another execution. Because the on-demand code execution system contains multiple execution environments, the system can efficiently handle a blocked execution by saving a state of the execution, and removing it from its execution environment. When a blocking dependency operation completes, the system can resume the blocked execution using the state information, in the same or different execution environment.

Claims

1. A system to manage blocking of code executions in an on-demand code execution system due to asynchronous operations, wherein the on-demand code execution system comprises a plurality of execution environments on which user-submitted code may execute, the system comprising:

a non-transitory data store configured to store state information regarding suspended executions of tasks on the on-demand code execution system, wherein individual tasks are associated with code executable to implement functionality corresponding to the individual tasks; and

one or more processors configured with computer-executable instructions to;

obtain instructions to execute a first task associated with first executable code;

begin execution of the first executable code within a first execution environment, wherein execution of the first executable code calls for execution of a first dependency operation;

after detecting that execution of the first executable code has become blocked awaiting completion of the first dependency operation;

determine that a predicted duration of blocking for the execution of the first executable code satisfied a threshold value;

generate state information for the execution of the first executable code;

store the generated state information in the non-transitory data store; and

remove the execution of the first executable code from the first execution environment; and

after detecting that the first dependency operation has completed;

select a second execution environment in which to resume execution of the first executable code; and

utilize the generated state information, as stored in the non-transitory data store, to resume execution of the first executable code in the second execution environment.

2. The system of claim 1, wherein the first execution environment is at least one of a virtual machine instance or a container.

3. The system of claim 1, wherein the second execution environment is a different execution environment from the first execution environment.

4. The system of claim 1, wherein the second execution environment is a regenerated version of the first execution environment.

5. The system of claim 1, wherein the state information for the execution of the first executable code comprises at least one of a virtual machine state, a container state, a state of memory associated with the execution, or a state of objects of the first executable code during the execution.

6. A computer-implemented method to manage blocking of code executions in an on-demand code execution system, wherein the on-demand code execution system comprises a plurality of execution environments on which user-submitted code may execute, the computer-implemented method comprising:

obtain instructions to execute first executable code within a first execution environment of the on-demand code execution system;

after detecting that execution of the first executable code is blocked awaiting completion of a first dependency operation;

generating state information for the execution of the first executable code;

storing the generated state information in a non-transitory data store distinct from the first execution environment; and

removing the execution of the first executable code from the first execution environment; and

after detecting that the first dependency operation has completed;

selecting a second execution environment in which to resume execution of the first executable code; and

utilizing the generated state information, as stored in the non-transitory data store, to resume execution of the first executable code in the second execution environment.

7. The computer-implemented method of claim 6 further comprising:

predicting a duration of blocking for the execution of the first executable code; and

determining that the predicted duration of blocking for the execution of the first executable code satisfied a threshold value.

8. The computer-implemented method of claim 7, wherein predicting a duration of blocking for the execution of the first executable code comprises:

predicting a first length of time between when the execution of the first executable code called for execution of the first dependency operation and when the execution of the first executable code requires completion of the first dependency operation;

predicting a second length of time required for execution of the first dependency operation to complete; and

assigning a difference between the first length of time and the second length of time as the predicted duration of blocking for the execution of the first executable code.

9. The computer-implemented method of claim 8, wherein the first length of time is predicted based on historical executions of the first executable code.

10. The computer-implemented method of claim 8, wherein the second length of time is predicted based on historical executions of the first dependency operation.

11. The computer-implemented method of claim 6, wherein selecting a second execution environment in which to resume execution of the first executable code comprises selecting the second execution environment based at least in part on the state information.

12. The computer-implemented method of claim 6, wherein the execution of the first executable code causes a call to execute an instance of the first dependency operation, and wherein detecting that the first dependency operation has completed comprises detecting that the instance of the first dependency operation has completed.

13. The computer-implemented method of claim 6, wherein a plurality of executions of the first executable code depend on completion of the first dependency operation, and wherein the method further comprises, on detecting that the first dependency operation has completed, selecting at least one of the plurality of executions of the first executable code to be resumed.

14. The computer-implemented method of claim 6 further comprising:

during execution of the first execution code, detecting a call for execution of a second dependency operation;

determining a deadline for the second dependency operation based at least in part on historical data regarding prior executions of the first executable code;

enqueuing the first dependency operation into the queue based at least in part on the deadline; and

processing the queue based at least in part on an available capacity of the on-demand code execution system to execute operations, wherein processing the queue comprises executing the first dependency operation.

15. Non-transitory computer-readable storage media including computer-executable instructions that, when executed by a computing system, cause the computing system to:

obtain instructions to execute first executable code within a first execution environment of an on-demand code execution system comprising a plurality of execution environments;

after detecting that execution of the first executable code has become blocked awaiting completion of a first dependency operation;

generate state information for the execution of the first executable code;

store the generated state information in a non-transitory data store distinct from the first execution environment; and

remove the execution of the first executable code from the first execution environment; and

after detecting that the first dependency operation has completed;

select a second execution environment in which to resume execution of the first executable code; and

utilize the generated state information, as stored in the non-transitory data store, to resume execution of the first executable code in the second execution environment.

16. The non-transitory computer-readable storage media of claim 15, wherein the computer-executable instructions further cause the computing system to:

predict a duration of blocking for the execution of the first executable code; and

determine that the predicted duration of blocking for the execution of the first executable code satisfied a threshold value.

17. The non-transitory computer-readable storage media of claim 16, wherein predicting the duration of blocking for the execution of the first executable code comprises:

predicting a first length of time between when the execution of the first executable code called for execution of the first dependency operation and when the execution of the first executable code requires completion of the first dependency operation;

predicting a second length of time required for execution of the first dependency operation to complete; and

assigning a difference between the first length of time and the second length of time as the predicted duration of blocking for the execution of the first executable code.

18. The non-transitory computer-readable storage media of claim 15, wherein at least one of the first or second lengths of time is predicted based on historical executions on the on-demand code execution system.

19. The non-transitory computer-readable storage media of claim 15, wherein the computer-executable instructions cause the computing system to select the second execution environment in which to resume execution of the first executable code based at least in part on the state information.

20. The non-transitory computer-readable storage media of claim 15, wherein the execution of the first executable code causes a call to execute an instance of the first dependency operation, and wherein detecting that the first dependency operation has completed comprises detecting that the instance of the first dependency operation has completed.

21. The non-transitory computer-readable storage media of claim 15, wherein a plurality of executions of the first executable code depend on completion of the first dependency operation, and wherein the computer-executable instructions cause the computing system to select at least one of the plurality of executions of the first executable code to be resumed.

22. The non-transitory computer-readable storage media of claim 15, wherein the computer-executable instructions further cause the computing system to:

during execution of the first execution code, detect a call for execution of a second dependency operation;

determine a deadline for the second dependency operation based at least in part on historical data regarding prior executions of the first executable code; and

execute the first dependency operation based at least in part on the deadline and an available capacity of the on-demand code execution system to execute operations.