Threading as a service

• US 9,600,312 B2
• Filed: 09/30/2014
• Issued: 03/21/2017
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen
• Primary Examiner(s): DAO, THUY CHAN
• Application Number:
  US14/502,992
• Publication Number:
  US 20160092252A1
• Time in Patent Office: 903 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A service manages a plurality of virtual machine instances for low latency execution of user codes. The plurality of virtual machine instances can be configured based on a predetermined set of configurations. One or more containers may be created within the virtual machine instances. In response to a request to execute user code, the service identifies a pre-configured virtual machine instance suitable for executing the user code. The service can allocate the identified virtual machine instance to the user, create a new container within an instance already allocated to the user, or re-use a container already created for execution of the user code. When the user code has not been activated for a time-out period, the service can invalidate allocation of the virtual machine instance destroy the container. The time from receiving the request to beginning code execution is less than a predetermined duration, for example, 100 ms.

Claims

1. A computer implemented method to process requests to execute user code on one or more virtual machine instances, the method comprising:

as implemented by one or more computing devices configured with specific computer-executable instructions,

providing a plurality of warmed virtual machine instances, each of the warmed virtual machine instances being unassigned to a specific user and containing a software component associated with a programming language;

subsequent to providing the plurality of warmed, unassigned virtual machine instances, receiving a request to execute a user code, the request comprising configuration information associated with executing the user code, wherein the request is received at a first time;

identifying a virtual machine instance from the plurality of warmed virtual machine instances based on the configuration information of the request, wherein the identified virtual machine instance contains a particular software component that is suitable to execute the user code;

associating the identified virtual machine instance with a user account associated with the request;

creating, within the associated virtual machine instance, a container to execute the user code;

loading the particular software component and the user code into the container; and

initiating execution of the user code by the container, wherein said initiating occurs at a second time, and

wherein a time period from the first time to the second time is less than a predetermined duration.

2. The method as recited in claim 1, wherein the predetermined duration is 100 ms.

3. The method as recited in claim 1, wherein creating the container further comprises:

receiving a plurality of requests to execute the user code;

creating a plurality of containers within the identified virtual machine instance, wherein each of the plurality of containers is configured based on the configuration information of the request; and

selecting one of the plurality of containers to execute the user code.

4. The method as recited in claim 1, wherein creating the container further comprises:

receiving a plurality of requests to execute a plurality of user codes;

creating a plurality of containers within the identified virtual machine instance, wherein each of the plurality of containers is configured based on configuration information of the requests; and

selecting one of the plurality of containers to execute the user code.

5. The method as recited in claim 1, wherein creating the at least one container further comprises:

determining resource requirements using the configuration information; and

creating at least one container having at least the resource requirements.

6. The method as recited in claim 1, wherein the software component comprises at least one of a runtime or one or more libraries.

7. The method as recited in claim 1, further comprising:

monitoring an activation history of the user code in the identified virtual machine instance; and

dissociating the identified virtual machine instance from the user account or destroying the container based on the activation history.

8. A system comprising:

a computing device comprising a processor coupled to a memory, the memory including specific instructions that upon execution configure the system to;

provide a plurality of warmed virtual machine instances, each of the warmed virtual machine instances being unassigned to a specific user and containing a software component associated with a programming language;

subsequent to providing the plurality of warmed, unassigned virtual machine instances, receive a request to execute a user code, the request comprising configuration information associated with executing the user code, wherein the request is received at a first time;

identify a virtual machine instance from the plurality of warmed virtual machine instances based on the configuration information of the request, wherein the identified virtual machine instance contains a particular software component that is suitable to execute the user code;

associate the identified virtual machine instance with a user account associated with the request;

create, within the associated virtual machine instance, a container to execute the user code;

load the particular software component and the user code into the container; and

initiate execution of the user code by the container, wherein said initiating occurs at a second time,

wherein a time period from the first time to the second time is less than a predetermined duration.

9. The system as recited in claim 8, wherein the predetermined duration is 100 ms.

10. The system as recited in claim 8, wherein the specific instructions further configure the system to provide the plurality of virtual machine instances before receiving the request to execute the user code.

11. The system as recited in claim 8, wherein the software component comprises at least one of a runtime or one or more libraries.

12. The system as recited in claim 8, wherein the specific instructions further configure the system to:

monitor an activation history of the user code in the identified virtual machine instance; and

dissociate the identified virtual machine instance from the user account or destroy the container based on the activation history.

13. The system as recited in claim 8, wherein the identified virtual machine instance comprises the container prior to receiving the request.

14. The system as recited in claim 13, wherein the container comprises the user code prior to receiving the request.

15. The system as recited in claim 13, wherein the specific instructions further configure the system to select the container based on resource requirements associated with the request.

16. The system as recited in claim 8, wherein the specific instructions further configure the system to create the container on the virtual machine instance after receiving the request.

17. A non-transitory, computer-readable storage medium storing computer-executable instructions that, when executed by a computer system, configure the computer system to;

provide a plurality of warmed virtual machine instances, each of the warmed virtual machine instances being unassigned to a specific user and containing a software component associated with a programming language;

subsequent to providing the plurality of warmed, unassigned virtual machine instances, receive a request to execute a user code, the request comprising configuration information associated with executing the user code, wherein the request is received at a first time;

identify a virtual machine instance from the plurality of warmed virtual machine instances based on the configuration information of the request, wherein the identified virtual machine instance contains a particular software component that is suitable to execute the user code;

associate the identified virtual machine instance with a user account associated with the request;

create, within the associated virtual machine instance, a container to execute the user code;

load the particular software component and the user code into the container; and

initiate execution of the user code by the container, wherein said initiating occurs at a second time,

wherein a time period from the first time to the second time is less than a predetermined duration.

18. The non-transitory, computer-readable storage medium as recited in claim 17, wherein the predetermined duration is 100 ms.

19. The non-transitory, computer-readable storage medium as recited in claim 17, wherein the instructions further configure the computer system to provide the plurality of virtual machine instances prior to receiving the request.

20. The non-transitory, computer-readable storage medium as recited in claim 17, wherein the software component comprises at least one of a runtime or one or more libraries.

21. The non-transitory, computer-readable storage medium as recited in claim 17, wherein the instructions further configure the computer system to

monitor an activation history of the user code in the identified virtual machine instance;

and

dissociate the identified virtual machine instance from the user account or destroy the container based on the activation history.

22. The non-transitory, computer-readable storage medium as recited in claim 17, wherein the identified virtual machine instance comprises the container prior to receiving the request.

23. The non-transitory, computer-readable storage medium as recited in claim 22, wherein the container comprises the user code prior to receiving the request.

24. The non-transitory, computer-readable storage medium as recited in claim 22, wherein the instructions further configure the computer system to select the container based on resource requirements associated with the request.

25. The non-transitory, computer-readable storage medium as recited in claim 17, wherein the instructions further configure the computer system to create the container on the virtual machine instance after receiving the request.

26. A computer implemented method for managing virtual machine instances, the method comprising:

providing a plurality of virtual machine instances, wherein each of the plurality of virtual machine instances contains at least one software component associated with at least one programming language;

receiving a request to execute a user code, the request comprising configuration information associated with the user code, wherein the request is received at a first time;

identifying a virtual machine instance from the plurality of virtual machine instances based on the configuration information of the request, wherein the identified virtual machine instance contains a particular software component that corresponds to the configuration information;

identifying a container having a first copy of the user code loaded thereon within the identified virtual machine instance;

causing the first copy of the user code loaded on the container to be executed based on the configuration information, wherein the first copy of the user code is caused to begin executing within a predetermined duration after the first time;

monitoring an activation history of the user code, wherein the activation history comprises information regarding execution of the user code in the identified virtual machine instance; and

dissociating the identified virtual machine instance from the user account or destroying the container based on the activation history.

27. The method as recited in claim 26, wherein the predetermined duration is 100 ms.

28. The method as recited in claim 26, wherein the method further comprises providing the plurality of warmed virtual machine instances before receiving the request.

29. The method as recited in claim 26, wherein the software component comprises at least one of a runtime or one or more libraries.

30. The method as recited in claim 26, further comprising:

determining, based on the activation history, that the identified virtual machine instance has not been used to execute the user code for a time period greater than a threshold time period; and

dissociating the identified virtual machine instance from the user account.

31. The method as recited in claim 26, wherein the identified virtual machine instance comprises the container prior to receiving the request.

32. The method as recited in claim 31, wherein the container comprises the user code prior to receiving the request.

33. The method as recited in claim 31, further comprising selecting the container based on resource requirements associated with the request.

34. The method as recited in claim 26, further comprising creating the container on the virtual machine instance after receiving the request.