Managing custom runtimes in an on-demand code execution system

• US 10,303,492 B1
• Filed: 12/13/2017
• Issued: 05/28/2019
• Est. Priority Date: 12/13/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen, Brooker, Marc John, Moffatt, Bryan Nicholas, Rodrigues dos Santos, Robison, Jayakar, Niranjan
• Primary Examiner(s): Nahar, Qamrun
• Application Number:
  US15/841,132
• Time in Patent Office: 531 Days
• Field of Search: None
• CPC Class Codes: G06F 8/315 Object-oriented languages
• CPC Class Codes: G06F 8/41 Compilation
• CPC Class Codes: G06F 9/44568 Immediately runnable code
• CPC Class Codes: G06F 9/45504 Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06F 9/547 Remote procedure calls [RPC]; Web services

Abstract

Systems and methods are described for managing provisioning of runtimes to execution environments within an on-demand code execution system. Different runtimes can include various software enabling execution of user-submitted code within the on-demand code execution system, potentially written in different programming languages. Each runtime can implement a pull-based request handler, such that when the runtime is provisioned within an execution environment, the runtime requests execution instructions from a management system on the on-demand code execution system. The management system can maintain a queue of execution instructions for each runtime, and can use a depth of each queue to manage the provisioning of environments with runtimes.

Claims

1. A system comprising:

a non-transitory data store including a plurality of runtimes, individual runtimes including software that, when provisioned into an execution environment on an on-demand code execution system, enable execution of user-submitted code within the execution environment;

one or more computing devices providing a plurality of execution environments of the on-demand code execution system;

a computing device implementing a management system, wherein the management system is configured to;

provision individual execution environments, of the plurality of execution environments, with a runtime from the plurality of runtimes;

maintain instruction queues for individual runtimes of the plurality of runtimes, wherein an instruction queue for an individual runtime enqueues one or more execution instructions awaiting distribution to an execution environment provisioned with the individual runtime, an execution instruction corresponding to an instruction to execute user-submitted code within an individual execution environment;

determine, based at least partly on a number of execution instructions within the instruction queue for a first runtime of the plurality of runtimes, a number of the plurality of execution environments to be provisioned with the first runtime; and

modify provisioning of the individual execution environments to cause the determined number of execution environments to be provisioned with the first runtime.

2. The system of claim 1, wherein the software of the individual runtimes comprises at least one of an interpreter, a compiler, a library, a software development kit, or a driver.

3. The system of claim 1, wherein the plurality of runtimes includes at least a first runtime enabling execution of code of a first programming language and a second runtime enabling execution of code within a second programming language.

4. The system of claim 1, wherein the first runtime is submitted to the on-demand code execution system by an end user of the on-demand code execution system.

5. The system of claim 1, wherein the individual execution environments, when provisioned with a runtime of the plurality of runtimes, are configured to transmit to the management system requests for execution instructions.

6. A computer-implemented method comprising:

provisioning individual execution environments, of a plurality of execution environments in an on-demand code execution system, with a runtime from a plurality of runtimes maintained at the on-demand code execution system, wherein individual runtimes include software that, when provisioned into an execution environment of the plurality of execution environments, enable execution of user-submitted code within the execution environment;

maintaining instruction queues for individual runtimes of the plurality of runtimes, wherein an instruction queue for an individual runtime enqueues one or more execution instructions awaiting distribution to an execution environment provisioned with the individual runtime, an execution instruction corresponding to an instruction to execute user-submitted code within an individual execution environment;

determining, based at least partly on a number of execution instructions within an instruction queue of a first runtime, of the plurality of runtimes, a number of the plurality of execution environments to be provisioned with the first runtime; and

modifying provisioning of the individual execution environments to cause the determined number of execution environments to be provisioned with the first runtime.

7. The computer-implemented method of claim 6, wherein the individual execution environments, when provisioned with a runtime of the plurality of runtimes, are configured to transmit to the management system requests for execution instructions, and wherein the method further comprises:

receiving a request for execution instructions from an execution environment provisioned with the first runtime;

identifying an execution instruction within the instruction queue for the first runtime;

transmitting the execution instruction to the execution environment provisioned with the first runtime; and

dequeue the execution instruction from the instruction queue for the runtime.

8. The computer-implemented method of claim 7, wherein the request is formatted according to the hypertext transmission protocol (HTTP).

9. The computer-implemented method of claim 7 further comprising implementing an application programming interface (API) through which the request is received.

10. The computer-implemented method of claim 9, wherein the API implements a representational state transfer (REST) architecture.

11. The computer implemented method of claim 7, wherein the instruction queues are maintained at at least a first computing device of the on-demand code execution system, and wherein the method further comprises routing the execution instruction to the first computing device based on applying a consistent hashing algorithm to an identifier of the identified runtime.

12. The computer implemented method of claim 6, wherein determining the number of the plurality of execution environments to be provisioned with the first runtime comprises:

determining that the number of execution instructions within the instruction queue of the first runtime exceeds a threshold value; and

determining an increase in the number of the plurality of execution environments to be provisioned with the first runtime.

13. The computer implemented method of claim 9, wherein the threshold value is at least one of an absolute threshold value or a threshold value relative to the number of execution instructions within one or more other instructions queues.

14. Non-transitory computer-readable media comprising computer-executable instructions that, when executed by a computing system, cause the computing system to:

maintain instruction queues for individual runtimes of a plurality of runtimes in an on-demand code execution system, wherein an instruction queue for an individual runtime enqueues one or more execution instructions awaiting distribution to an execution environment, among a plurality of execution environments on the on-demand code execution system, provisioned with the individual runtime, the one or more execution instructions corresponding to instructions to execute user-submitted code within the execution environment, wherein individual runtimes include software that, when provisioned into the execution environment, enable execution of user-submitted code within the execution environment;

determine, based at least partly on a number of execution instructions within an instruction queue of a first runtime of the plurality of runtimes, a number of the plurality of execution environments to be provisioned with the first runtime; and

cause the determined number of execution environments to be provisioned with the first runtime.

15. The non-transitory computer-readable media of claim 14, wherein causing the determined number of execution environments to be provisioned with the first runtime comprises causing a reduction in the number of execution environments provisioned with a second runtime.

16. The non-transitory computer-readable media of claim 14, wherein the instructions further cause the computing system to identify the second runtime based at least in part on the instruction queue of the second runtime.

17. The non-transitory computer-readable media of claim 14, wherein causing the determined number of execution environments to be provisioned with the first runtime comprises causing a reduction in the number of execution environments provisioned with a second runtime.

18. The non-transitory computer-readable media of claim 14, wherein determining the number of the plurality of execution environments to be provisioned with the first runtime comprises:

determining that the number of execution instructions within the instruction queue of the first runtime is lower than a threshold value; and

determining a decrease in the number of the plurality of execution environments to be provisioned with the first runtime.

19. The non-transitory computer-readable media of claim 14, wherein the instructions further cause the computing system to manage a number of execution environments, within the plurality of execution environments, independently of provisioning the plurality of execution environments with runtimes of the plurality of runtimes.

20. The non-transitory computer-readable media of claim 14, wherein the instructions further cause the computing system to generate the plurality of execution environments.