THREADING AS A SERVICE

• US 20170192804A1
• Filed: 03/20/2017
• Published: 07/06/2017
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen
• Granted Patent: US 10,140,137 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A service manages a plurality of virtual machine instances for low latency execution of user codes. The plurality of virtual machine instances can be configured based on a predetermined set of configurations. One or more containers may be created within the virtual machine instances. In response to a request to execute user code, the service identifies a pre-configured virtual machine instance suitable for executing the user code. The service can allocate the identified virtual machine instance to the user, create a new container within an instance already allocated to the user, or re-use a container already created for execution of the user code. When the user code has not been activated for a time-out period, the service can invalidate allocation of the virtual machine instance destroy the container. The time from receiving the request to beginning code execution is less than a predetermined duration, for example, 100 ms.

Claims

1-21. -21. (canceled)

22. A computer-implemented method to process requests to execute user code on one or more virtual machine instances, the method comprising:

as implemented by one or more computing devices configured with specific computer-executable instructions,

providing a plurality of warmed virtual machine instances, each of the warmed virtual machine instances containing a software component associated with a programming language;

subsequent to providing the plurality of warmed virtual machine instances, receiving a request to execute a user code, the request comprising configuration information for executing the user code;

identifying a virtual machine instance from the plurality of warmed virtual machine instances based on the configuration information, wherein the virtual machine instance contains a particular software component usable to execute the user code;

creating, within the virtual machine instance, a container in which to execute the user code;

loading the user code into the container within the virtual machine instance; and

executing the user code in the container within the virtual machine instance.

23. The method of claim 22, wherein one or both of the virtual machine instance and the container are associated with a user account associated with the request.

24. The method of claim 22, further comprising:

receiving a plurality of requests to execute the user code;

creating a plurality of containers within the virtual machine instance, wherein each of the plurality of containers is configured based on configuration information of one of the plurality of requests; and

selecting one of the plurality of containers to execute the user code.

25. The method of claim 22, further comprising:

determining one or more resource requirements based on the configuration information; and

creating the container such that the container meets the one or more resource requirements.

26. The method of claim 22, wherein the software component comprises one or both of a runtime and a library.

27. The method of claim 22, further comprising keeping the virtual machine instance running even after completion of the execution of the user code until another request to execute the user code is received.

28. The method of claim 22, further comprising keeping the virtual machine instance running even after completion of the execution of the user code until a threshold amount of time is elapsed without the virtual machine instance being used.

29. The method of claim 22, wherein the request includes one of (i) an identifier associated with the user code or (ii) a location of the user code.

30. The method of claim 22, further comprising:

determining one or more arguments included in the request; and

executing the user code using the one or more arguments.

31. A system for processing requests to execute user code on one or more virtual machine instances, the system comprising:

one or more processors;

one or more memories, the one or more memories having stored thereon instructions, which, when executed by the one or more processors, cause the one or more processors to;

provide a plurality of warmed virtual machine instances, each of the warmed virtual machine instances containing a software component associated with a programming language;

subsequent to providing the plurality of warmed virtual machine instances, receive a request to execute a user code, the request comprising configuration information for executing the user code;

identify a virtual machine instance from the plurality of warmed virtual machine instances based on the, configuration information, wherein the virtual machine instance contains a particular software component usable to execute the user code;

create, within the virtual machine instance, a container in which to execute the user code;

load the user code into the container within the virtual machine instance; and

execute the user code in the container within the virtual machine instance.

32. The system of claim 31, wherein one or both of the virtual machine instance and the container are associated with a user account associated with the request.

33. The system of claim 31, wherein the software component comprises one or both of a runtime and a library.

34. The system of claim 31, further comprising instructions to keep the virtual machine instance running even after completion of the execution of the user code until another request to execute the user code is received.

35. The system of claim 31, further comprising instructions to keep the virtual machine instance running even after completion of the execution of the user code until a threshold amount of time is elapsed without the virtual machine instance being used.

36. The system of claim 31, wherein the request includes one of (i) an identifier associated with the user code or (ii) a location of the user code.

37. Non-transitory physical computer storage storing computer-executable instructions, which, when executed by a computer system, cause the computer system to:

provide a plurality of warmed virtual machine instances, each of the warmed virtual machine instances containing a software component associated with a programming language;

subsequent to providing the plurality of warmed virtual machine instances, receive a request to execute a user code, the request comprising configuration information for executing the user code;

identify a virtual machine instance from the plurality of warmed virtual machine instances based on the configuration information, wherein the virtual machine instance contains a particular software component usable to execute the user code;

create, within the virtual machine instance, a container in which to execute the user code;

load the user code into the container within the virtual machine instance; and

execute the user code in the container within the virtual machine instance.

38. The non-transitory physical computer storage of claim 37, wherein one or both of the virtual machine instance and the container are associated with a user account associated with the request.

39. The non-transitory physical computer storage of claim 37, wherein the software component comprises one or both of a runtime and a library.

40. The non-transitory physical computer storage of claim 37, further comprising keeping the virtual machine instance running even after completion of the execution of the user code until another request to execute the user code is received.

41. The non-transitory physical computer storage of claim 37, further comprising keeping the virtual machine instance running even after completion of the execution of the user code until a threshold amount of time is elapsed without the virtual machine instance being used.