Acquisition and maintenance of compute capacity

• US 10,067,801 B1
• Filed: 12/21/2015
• Issued: 09/04/2018
• Est. Priority Date: 12/21/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen
• Primary Examiner(s): Sun, Charlie
• Application Number:
  US14/977,524
• Time in Patent Office: 988 Days
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/45575 Starting, stopping, suspending or resuming virtual machine instances
• CPC Class Codes: G06F 2209/5011 Pool
• CPC Class Codes: G06F 2209/5013 Request control
• CPC Class Codes: G06F 2209/506 Constraint
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A system for providing low-latency computational capacity from a virtual compute fleet is provided. The system may be configured to maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises a first pool comprising a first sub-pool of virtual machine instances and a second sub-pool of virtual machine instances, and a second pool comprising virtual machine instances used for executing one or more program codes thereon. The first sub-pool and/or the second sub-pool may be associated with one or more users of the system. The system may be further configured to process code execution requests and execute program codes on the virtual machine instances of the first or second sub-pool.

Claims

1. A system for providing low-latency computational capacity, the system comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises;

a warming pool comprising a first sub-pool of virtual machine instances and a second sub-pool of virtual machine instances, wherein the first sub-pool of virtual machine instances are associated with one or more fixed resource constraints and the second sub-pool of virtual machine instances are associated with one or more variable resource constraints; and

an active pool comprising virtual machine instances executing one or more program codes thereon;

receive a first request to execute a first program code associated with a first user, the first request including an identity of a user account associated with the first user and one or more parameters for executing the first program code;

determine, based on the identity of the user account, that the first program code is to be executed using a virtual machine instance associated with the second sub-pool that satisfies a first variable resource constraint;

in response to a determination that the second sub-pool lacks a virtual machine instance satisfying the first variable resource constraint, wait until a first virtual machine instance satisfying the first variable resource constraint becomes available in the second sub-pool;

cause the first virtual machine instance in the second sub-pool to be added to the active pool; and

cause the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance.

2. The system of claim 1, wherein the first sub-pool is configured to provide guaranteed availability, and the second sub-pool is configured to provide reduced availability prone to interruptions.

3. The system of claim 1, wherein the one or more fixed resource constraints include a fixed cost per unit of compute capacity, and the one or more variable resource constraints include a variable cost per unit of compute capacity based on market demand.

4. The system of claim 1, wherein the virtual compute system is further configured to automatically acquire additional virtual machine instances to be added to the second sub-pool from an instance provisioning service based on a user policy associated with the first user.

5. A system for providing low-latency computational capacity, the system comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises;

a first pool comprising virtual machine instances associated with one or more variable resource constraints; and

a second pool comprising virtual machine instances executing one or more program codes thereon;

receive a first request to execute a first program code associated with a first user, the first request including user account information and one or more parameters for executing the first program code;

determine, based on the user account information, that the first program code is to be executed using a virtual machine instance associated with the first pool that satisfies a first variable resource constraint;

subsequent to a first virtual machine instance satisfying the first variable resource constraint becoming available in the first pool, cause the first virtual machine instance in the first pool to be added to the second pool; and

cause the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance.

6. The system of claim 5, wherein the virtual compute system is further configured to maintain a third pool comprising virtual machine instances associated with one or more fixed resource constraints, wherein the third pool is configured to provide guaranteed availability, and the first pool is configured to provide reduced availability prone to interruptions.

7. The system of claim 6, wherein the one or more fixed resource constraints include a fixed cost per unit of compute capacity, and the one or more variable resource constraints include a variable cost per unit of compute capacity based on market demand.

8. The system of claim 5, wherein the virtual compute system is further configured to automatically acquire additional virtual machine instances to be added to the first pool from an instance provisioning service based on a user policy associated with the first user.

9. The system of claim 5, wherein the virtual compute system is further configured to:

determine a resource constraint associated with the first user;

acquire additional compute capacity that satisfies the resource constraint associated with the first user for a duration that is greater than a maximum duration associated with the first program code; and

execute the first program code using the acquired additional capacity.

10. The system of claim 5, wherein the virtual compute system is further configured to:

in response to determining that the first pool lacks compute capacity sufficient to process the first request, determine whether a policy associated with the first pool allows the virtual compute system to wait for additional compute capacity to be acquired; and

in response to determining that the policy associated with the first pool allows the virtual compute system to wait for additional compute capacity to be acquired, wait for additional compute capacity to be acquired.

11. The system of claim 5, wherein the virtual compute system is further configured to:

determine a time window during which the first request should be processed;

acquire additional compute capacity that satisfies one or more resource criteria in the determined time window; and

execute the first program code using the acquired additional capacity.

12. The system of claim 6, wherein the virtual compute system is further configured to:

receive a second request to execute the first program code;

determine whether the second request is a time-sensitive request; and

perform one of (i) in response to determining that the second request is not a time-sensitive request, execute the first program code using a virtual machine instance associated with the first pool, or (ii) in response to determining that the second request is a time-sensitive request, execute the first program code using a virtual machine instance associated with the third pool, wherein the third pool is associated with the first user.

13. A computer-implemented method comprising:

as implemented by one or more computing devices configured with specific executable instructions,

maintaining a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises;

a first pool comprising virtual machine instances associated with one or more variable resource constraints; and

a second pool comprising virtual machine instances executing one or more program codes thereon;

receiving a first request to execute a first program code associated with a first user, the first request including user account information and one or more parameters for executing the first program code;

determining, based on the user account information, that the first program code is to be executed using a virtual machine instance associated with the first pool that satisfies a first variable resource constraint;

subsequent to a first virtual machine instance satisfying the first variable resource constraint becoming available in the first pool, causing the first virtual machine instance in the first pool to be added to the second pool; and

causing the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance.

14. The method of claim 13, further comprising maintaining a third pool comprising virtual machine instances associated with one or more fixed resource constraints, wherein the third pool is configured to provide guaranteed availability, and the first pool is configured to provide reduced availability prone to interruptions.

15. The method of claim 14, wherein the one or more fixed resource constraints include a fixed cost per unit of compute capacity, and the one or more variable resource constraints include a variable cost per unit of compute capacity based on market demand.

16. The method of claim 13, further comprising automatically acquiring additional virtual machine instances to be added to the first pool from an instance provisioning service based on a user policy associated with the first user.

17. The method of claim 13, further comprising:

determining a resource constraint associated with the first user;

acquiring additional compute capacity that satisfies the resource constraint associated with the first user for a duration that is greater than a maximum duration associated with the first program code; and

executing the first program code using the acquired additional capacity.

18. The method of claim 13, further comprising:

in response to determining that the first pool lacks compute capacity sufficient to process the first request, determining whether a policy associated with the first pool allows waiting for additional compute capacity to be acquired; and

in response to determining that the policy associated with the first pool allows waiting for additional compute capacity to be acquired, waiting for additional compute capacity to be acquired.

19. The method of claim 13, further comprising:

determining a time window during which the first request should be processed;

acquiring additional compute capacity that satisfies one or more resource criteria in the determined time window; and

executing the first program code using the acquired additional capacity.

20. The method of claim 14, wherein the virtual compute system is further configured to:

receive a second request to execute the first program code;

determining whether the second request is a time-sensitive request; and

performing one of (i) in response to determining that the second request is not a time-sensitive request, executing the first program code using a virtual machine instance associated with the first pool, or (ii) in response to determining that the second request is a time-sensitive request, executing the first program code using a virtual machine instance associated with the third pool, wherein the third pool is associated with the first user.