Acquisition and maintenance of dedicated, reserved, and variable compute capacity
First Claim
1. A system for providing low-latency computational capacity, the system comprising:
- a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;
maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises;
a first sub-pool of virtual machine instances waiting to be assigned to one of a plurality of user accounts of the virtual compute system;
a second sub-pool of virtual machine instances assigned, before a request associated with a first user account of the plurality of user accounts is received by the virtual compute system, to the first user account and having one or more software components specific to the first user account loaded thereon; and
an active pool comprising virtual machine instances executing one or more program codes thereon;
receive the request to execute a first program code using compute capacity provided by one of the plurality of virtual machine instances, the request including an identity associated with the first user account and one or more parameters for executing the first program code;
determine, based on the identity included in the request to execute the first program code, that the first program code is to be executed using compute capacity provided by one of the virtual machine instances in the second sub-pool;
determine that the second sub-pool lacks compute capacity sufficient to execute the first program code;
in response to determining that the second sub-pool lacks compute capacity sufficient to execute the first program code, determine that a policy associated with the second sub-pool allows the virtual compute system to wait for additional compute capacity sufficient to execute the first program code to become available;
in response to determining that the policy allows the virtual compute system to wait, wait for additional compute capacity sufficient to execute the first program code to become available in the second sub-pool;
cause a first virtual machine instance in the second sub-pool having compute capacity sufficient to execute the first program code to be added to the active pool; and
cause the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance.
1 Assignment
0 Petitions
Accused Products
Abstract
A system for providing low-latency computational capacity from a virtual compute fleet is provided. The system may be configured to maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises a first pool comprising a first sub-pool of virtual machine instances and a second sub-pool of virtual machine instances, and a second pool comprising virtual machine instances used for executing one or more program codes thereon. The first sub-pool and/or the second sub-pool may be associated with one or more users of the system. The system may be further configured to process code execution requests and execute program codes on the virtual machine instances of the first or second sub-pool.
375 Citations
21 Claims
-
1. A system for providing low-latency computational capacity, the system comprising:
a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least; maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises; a first sub-pool of virtual machine instances waiting to be assigned to one of a plurality of user accounts of the virtual compute system; a second sub-pool of virtual machine instances assigned, before a request associated with a first user account of the plurality of user accounts is received by the virtual compute system, to the first user account and having one or more software components specific to the first user account loaded thereon; and an active pool comprising virtual machine instances executing one or more program codes thereon; receive the request to execute a first program code using compute capacity provided by one of the plurality of virtual machine instances, the request including an identity associated with the first user account and one or more parameters for executing the first program code; determine, based on the identity included in the request to execute the first program code, that the first program code is to be executed using compute capacity provided by one of the virtual machine instances in the second sub-pool; determine that the second sub-pool lacks compute capacity sufficient to execute the first program code; in response to determining that the second sub-pool lacks compute capacity sufficient to execute the first program code, determine that a policy associated with the second sub-pool allows the virtual compute system to wait for additional compute capacity sufficient to execute the first program code to become available; in response to determining that the policy allows the virtual compute system to wait, wait for additional compute capacity sufficient to execute the first program code to become available in the second sub-pool; cause a first virtual machine instance in the second sub-pool having compute capacity sufficient to execute the first program code to be added to the active pool; and cause the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance. - View Dependent Claims (2, 3, 4, 5)
-
6. A system for providing low-latency computational capacity, the system comprising:
a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least; maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises; a first pool comprising virtual machine instances assigned to a first user account before a request associated with the first user account is received by the virtual compute system; and a second pool comprising virtual machine instances executing one or more program codes thereon; receive the request to execute a first program code using compute capacity provided by one of the plurality of virtual machine instances, the request including first user account information and one or more parameters for executing the first program code; determine, based on the first user account information, that the first program code is to be executed using compute capacity provided by a virtual machine instance in the first pool; determine that the first pool lacks compute capacity sufficient to execute the first program code; in response to determining that the first pool lacks compute capacity sufficient to execute the first program code, determine that a policy associated with the first pool allows the virtual compute system to wait for additional compute capacity sufficient to execute the first program code to become available; in response to determining that the policy allows the virtual compute system to wait, wait for additional compute capacity sufficient to execute the first program code to become available in the first pool; cause a first virtual machine instance in the first pool having compute capacity sufficient to execute the first program code to be added to the second pool; and cause the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance. - View Dependent Claims (7, 8, 9, 10, 11, 12, 13)
-
14. A computer-implemented method comprising:
as implemented by one or more computing devices configured with specific executable instructions, maintaining a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprises; a first pool comprising virtual machine instances assigned to a first user account before a code execution request associated with the first user account is received by the virtual compute system; and a second pool comprising virtual machine instances executing one or more program codes thereon; receiving the request to execute a first program code using compute capacity provided by one of the plurality of virtual machine instances, the request including first user account information and one or more parameters for executing the first program code; determining, based on the first user account information, that the first program code is to be executed using compute capacity provided by a virtual machine instance in the first pool; determining that the first pool lacks compute capacity sufficient to execute the first program code; in response to determining that the first pool lacks compute capacity sufficient to execute the first program code, determining that a policy associated with the first pool allows the virtual compute system to wait for additional compute capacity sufficient to execute the first program code to become available; in response to determining that the policy allows the virtual compute system to wait, waiting for additional compute capacity sufficient to execute the first program code to become available in the first pool; causing a first virtual machine instance in the first pool having compute capacity sufficient to execute the first program code to be added to the second pool; and causing the first program code to be executed, using the one or more parameters, in a container created on the first virtual machine instance. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21)
Specification