Automatic management of low latency computational capacity
First Claim
Patent Images
1. A system for providing low-latency computational capacity from a virtual compute fleet, the system comprising:
- an electronic data store configured to store at least a program code of a user; and
a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions, said virtual compute system in communication with the data store, and configured to at least;
maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprise;
a warming pool comprising virtual machine instances having one or more software components loaded thereon; and
an active pool comprising virtual machine instances assigned to one or more users,wherein the virtual compute system is configured to add a first number of virtual machine instances to the warming pool in response to a first virtual machine instance being removed from the warming pool;
monitor incoming code execution requests to execute program codes on the virtual compute system, the incoming code execution requests comprising information usable for identifying the respective program codes to be executed on the virtual compute system;
determine a current rate at which the incoming code execution requests are received;
determine whether the current rate at which the incoming code execution requests are received satisfies a threshold condition for adjusting the warming pool; and
in response to determining that the current rate at which the incoming code execution requests are received satisfies the threshold condition, adjust a rate at which additional virtual machine instances are added to the warming pool such that a second number of virtual machine instances are added to the warming pool in response to a second virtual machine instance being removed from the warming pool, wherein the second number and is different from the first number and determined based on the current rate at which the incoming code execution requests are received.
1 Assignment
0 Petitions
Accused Products
Abstract
A system for providing automatic management of low latency computational capacity is provided. The system may be configured to maintain a plurality of virtual machine instances. The system may be further configured to identify a trend in incoming code execution requests to execute program code on a virtual compute system, determine, based on the identified trend, that the plurality of virtual machine instances should be adjusted, and adjust the plurality of virtual machine instances based on the identified trend.
324 Citations
22 Claims
-
1. A system for providing low-latency computational capacity from a virtual compute fleet, the system comprising:
-
an electronic data store configured to store at least a program code of a user; and a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions, said virtual compute system in communication with the data store, and configured to at least; maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprise; a warming pool comprising virtual machine instances having one or more software components loaded thereon; and an active pool comprising virtual machine instances assigned to one or more users, wherein the virtual compute system is configured to add a first number of virtual machine instances to the warming pool in response to a first virtual machine instance being removed from the warming pool; monitor incoming code execution requests to execute program codes on the virtual compute system, the incoming code execution requests comprising information usable for identifying the respective program codes to be executed on the virtual compute system; determine a current rate at which the incoming code execution requests are received; determine whether the current rate at which the incoming code execution requests are received satisfies a threshold condition for adjusting the warming pool; and in response to determining that the current rate at which the incoming code execution requests are received satisfies the threshold condition, adjust a rate at which additional virtual machine instances are added to the warming pool such that a second number of virtual machine instances are added to the warming pool in response to a second virtual machine instance being removed from the warming pool, wherein the second number and is different from the first number and determined based on the current rate at which the incoming code execution requests are received. - View Dependent Claims (2, 3)
-
-
4. A system, comprising:
a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least; maintain a pool of virtual machine instances on one or more physical computing devices such that a first number of virtual machine instances are added to the plurality pool of virtual machine instances in response to a first virtual machine instance being removed from the pool, determine a current rate at which incoming code execution requests are received; determine whether the current rate at which the incoming code execution requests are received satisfies a threshold condition for adjusting the pool of virtual machine instances; and in response to determining that the current rate at which the incoming code execution requests are received satisfies the threshold condition, adjust a rate at which additional virtual machine instances are added to the pool such that a second number of virtual machine instances are added to the pool in response to a second virtual machine instance being removed from the pool, wherein the second number is different from the first number and determined based on the current rate at which the incoming code execution requests are received. - View Dependent Claims (5, 6, 7, 8, 9, 10, 11, 12)
-
13. A computer-implemented method comprising:
as implemented by one or more computing devices configured with specific executable instructions, maintaining a pool of virtual machine instances on one or more physical computing devices such that a first number of virtual machine instances are added to the plurality pool of virtual machine instances in response to a first virtual machine instance being removed from the pool; determining a current rate at which incoming code execution requests are received; determining whether the current rate at which the incoming code execution requests are received satisfies a threshold condition for adjusting the pool of virtual machine instances; and in response to determining that the current rate at which the incoming code execution requests are received satisfies the threshold condition, adjusting a rate at which additional virtual machine instances are added to the pool such that a second number of virtual machine instances are added to the pool in response to a second virtual machine instance being removed from the pool, wherein the second number is different from the first number and determined based on the current rate at which the incoming code execution requests are received. - View Dependent Claims (14, 15, 16, 17)
-
18. Non-transitory physical computer storage storing computer executable instructions that, when executed by one or more computing devices, configure the one or more computing devices to:
-
maintain a pool of virtual machine instances on one or more physical computing devices such that a first number of virtual machine instances are added to the pool of virtual machine instances in response to a first virtual machine instance being removed from the pool; determine a current rate at which incoming code execution requests are received; determine whether the current rate at which the incoming code execution requests are received satisfies a threshold condition for adjusting the plurality pool of virtual machine instances; and in response to determining that the current rate at which the incoming code execution requests are received satisfies the threshold condition, adjust a rate at which additional virtual machine instances are added to the pool such that a second number of virtual machine instances are added to the pool in response to a second virtual machine instance being removed from the pool, wherein the second number is different from the first number and determined based on the current rate at which the incoming code execution requests are received. - View Dependent Claims (19, 20, 21, 22)
-
Specification