Code execution request routing

• US 9,910,713 B2
• Filed: 12/21/2015
• Issued: 03/06/2018
• Est. Priority Date: 12/21/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wisniewski, Scott Daniel, Brooker, Marc John, Wagner, Timothy Allen
• Primary Examiner(s): KIM, DONG U
• Application Number:
  US14/977,544
• Publication Number:
  US 20170177413A1
• Time in Patent Office: 806 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2209/5011 Pool
• CPC Class Codes: G06F 2209/5013 Request control
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5033 considering data affinity
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A system for providing low-latency compute capacity is provided. The system may be configured to route incoming code execution requests based on user indications to use specific containers running on a plurality of virtual machine instances. The system may be configured to process a code execution request, identify, based on the user indication that a specific container previously used to handle a request of similar type is to be used for handling the code execution request, and cause the code execution request to be handled using the specific container.

Claims

1. A system for providing low-latency computational capacity, the system comprising:

an external database service configured to store a plurality of key-value pairs;

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions, the virtual compute system in communication with the external database service and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises;

a warming pool comprising virtual machine instances having one or more software components loaded thereon and waiting to be assigned to a user; and

an active pool comprising virtual machine instances currently assigned to one or more users;

process a first request to execute a first program code associated with a first user on the virtual compute system, the first request including an identity of the first user, and one or more first execution parameters to be used for executing the first program code;

execute the first program code in a first container created on a first virtual machine instance in the active pool using the one or more first execution parameters, the execution of the program code causing one or more key-value pairs of the plurality of key-value pairs to be retrieved from the external database service and stored locally within the first container;

process a second request to execute the first program code on the virtual compute system, the second request including the identity of the first user, one or more second execution parameters to be used for executing the first program code, and a user indication that the second request is to be handled using the first container previously used for handling the first request; and

based on the user indication that the second request is to be handled using the first container, execute the first program code in the first container using the one or more second execution parameters, the execution of the first program code accessing the one or more key-value pairs locally stored within the first container such that a latency associated with the execution of the first program code based on the one or more second execution parameters is reduced.

2. The system of claim 1, wherein the virtual compute system is configured to identify the first container by calculating a hash value associated with the user indication and performing a lookup of the calculated hash value in a mapping table maintained by the virtual compute system, the mapping table mapping one or more hash values to specific containers running on the plurality of virtual machine instances.

3. The system of claim 1, wherein the execution of the first program code using the one or more second execution parameters omits at least one retrieval of data from the external database service based on the one or more key-value pairs locally stored within the first container.

4. The system of claim 1, wherein the virtual compute system is further configured to keep the first container running even after the execution of the first program code using the one or more second execution parameters has been completed such that subsequent requests providing indications to use the first container can be handled using the first container.

5. A system comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices;

process a first request to execute a first program code on the virtual compute system, the first request including account information, one or more parameters to be used for executing the first program code, and an indication that the first program code is to be executed based on data associated with a prior execution of the first program code;

identify a first container storing first data associated with one or more computing operations performed by a specific prior execution of the first program code, wherein the first container is on one of the plurality of virtual machine instances; and

cause the first program code to be executed in the first container based on the first data and without subsequently performing the one or more computing operations previously performed by the specific prior execution of the first program code.

6. The system of claim 5, wherein the virtual compute system is further configured to maintain a mapping table that maps one or more values associated with incoming code execution requests to specific containers running on the plurality of virtual machine instances.

7. The system of claim 5, wherein the one or more computing operations are configured to retrieve the first data from an external service that is in communication with the virtual compute system and cause the first data to be stored in local storage within the first container.

8. The system of claim 5, wherein the virtual compute system is further configured to:

determine that first container contains cached data associated with the specific prior execution of the first program code; and

cause the cached data to be presented to the execution of the first program code in the first container, wherein the execution of the first program code omits at least one computing operation based on the cached data.

9. The system of claim 5, wherein the first container is on a virtual machine instance having one or more instance-specific computing resources that are accessible by one or more containers running on the virtual machine instance.

10. The system of claim 5, wherein the indication comprises a parameter that causes the virtual compute system to route the first request to a specific container.

11. The system of claim 5, wherein the virtual compute system is further configured to route incoming code execution requests including the same indication to the same container.

12. The system of claim 5, wherein the virtual compute system is further configured to automatically execute a shutdown program code associated with the first container when in response to a shutdown of the first container.

13. A computer-implemented method comprising:

as implemented by one or more computing devices configured with specific executable instructions,

maintaining a plurality of virtual machine instances on one or more physical computing devices;

processing a first request to execute a first program code on a virtual compute system, the first request including account information, one or more parameters to be used for executing the first program code, and an indication that the first program code is to be executed based on data associated with a prior execution of the first program code;

identifying a first container storing first data associated with one or more computing operations performed by a specific prior execution of the first program code, wherein the first container is on one of the plurality of virtual machine instances; and

causing the first program code to be executed in the first container based on the first data and without subsequently performing the one or more computing operations previously performed by the specific prior execution of the first program code.

14. The method of claim 13, further comprising maintaining a mapping table that maps one or more values associated with incoming code execution requests to specific containers running on the plurality of virtual machine instances.

15. The method of claim 13, wherein the one or more computing operations are configured to retrieve the first data from an external service that is in communication with the virtual compute system and cause the first data to be stored in local storage within the first container.

16. The method of claim 13, further comprising:

determining that the first container contains cached data associated with the specific prior execution of the first program code; and

causing the cached data to be presented to the execution of the first program code in the first container, wherein the execution of the first program code omits the at least one computing operation based on the cached data.

17. The method of claim 13, wherein the first container is on a virtual machine instance having one or more instance-specific computing resources that are accessible by one or more containers running on the virtual machine instance.

18. The method of claim 13, wherein the indication comprises a parameter that causes the virtual compute system to route the first request to a specific container.

19. The method of claim 13, further comprising routing incoming code execution requests including the same indication to the same container.

20. The method of claim 13, further comprising automatically executing a shutdown program code associated with the first container in response to a shutdown of the first container.