Automatic management of resource sizing

• US 9,413,626 B2
• Filed: 12/05/2014
• Issued: 08/09/2016
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Reque, Sean Philip, Manwaring, Derek Steven, Thomas, Dylan Chandler, Wagner, Timothy Allen, Zhao, Xin
• Primary Examiner(s): Dao, Thuy
• Assistant Examiner(s): Smith, Cheneca
• Application Number:
  US14/562,577
• Publication Number:
  US 20160164762A1
• Time in Patent Office: 613 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/808 User-type aware
• CPC Class Codes: H04L 47/83 based on usage prediction

Abstract

A system for providing automatic resource resizing is provided. The system may be configured to maintain a plurality of virtual machine instances. The system may be further configured to receive a request to execute a program code and allocate computing resources for executing the program code on one of the virtual machine instances. The amount of resources allocated for executing the program code may be specified by the request and adjusted as needed.

Claims

1. A computer-implemented method comprising:

as implemented by one or more computing devices configured with specific executable instructions,

maintaining a plurality of virtual machine instances on one or more physical computing devices;

receiving a plurality of requests to execute a program code associated with a user on a virtual compute system, the request including information indicating the program code, the user associated with the program code, and a first amount of a computing resource available on the virtual compute system to be used to execute the program code;

selecting a virtual machine instance from the plurality of virtual machine instances to be used to execute the program code, the selected virtual machine instance having at least the first amount of the computing resource indicated in the plurality of requests;

creating one or more containers in the selected virtual machine instance, the one or more containers each having the first amount of the computing resource indicated in the plurality of requests;

causing the program code associated with the user to be loaded onto the one or more containers and executed in the one or more containers;

monitoring a usage of the computing resource by the program code during the executions of the program code in the one or more containers created in the selected virtual machine instance;

determining, based on the usage of the computing resource by the program code, that the first amount indicated in the plurality of requests has satisfied a resource adjustment condition for a threshold number of the executions in the one or more containers;

in response to receiving additional requests including information indicating the program code, the user associated with the program code, and the first amount of the computing resource to be used to execute the program code, creating one or more additional containers in the selected virtual machine instance such that the one or more additional containers each have a second amount of the computing resource that is different from the first amount indicated in the additional requests; and

causing the program code to be executed in the one or more additional containers having the second amount of the computing resource.

2. The computer-implemented method of claim 1, further comprising:

determining the first amount based on a user-specified amount indicated in the request or other configuration information associated with the request, the program code, or the user to execute the program code on the virtual compute system;

comparing the usage of the computing resource by the program code to the user-specified amount indicated in the request; and

providing an indication of how the first amount should be adjusted.

3. The computer-implemented method of claim 2, wherein providing the indication comprises one of:

(a) indicating that the first amount should be increased;

(b) indicating that the first amount should be increased by a first adjustment amount;

(c) indicating that the first amount should be decreased;

or (d) indicating that the first amount should be decreased by a second adjustment amount.

4. The computer-implemented method of claim 1, further comprising determining the second amount based on the usage of the computing resource by the program code.

5. The computer-implemented method of claim 4, wherein determining the second amount comprises determining the second amount such that the second amount is within a threshold range specified by a user associated with the program code.

6. The computer-implemented method of claim 1, further comprising allowing the usage of the computing resource by the program code to exceed the first amount by an overage amount that is less than a maximum overage amount.

7. A system adapted to provide low-latency computational capacity from a virtual compute fleet, the system comprising:

an electronic data store configured to store at least a program code of a user; and

a virtual compute system comprising one or more hardware computing devices adapted to execute specific computer-executable instructions, said virtual compute system in communication with the electronic data store, and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices, wherein the plurality of virtual machine instances comprise a warming pool comprising virtual machine instances having one or more software components loaded thereon and waiting to be assigned to a user, and an active pool comprising virtual machine instances currently assigned to one or more users;

receive a plurality of requests to execute a program code associated with a user on the virtual compute system, the plurality of requests each specifying the program code, the user associated with the program code, and a first amount of a computing resource available on the virtual compute system to be used to execute the program code;

select from the warming pool or the active pool a virtual machine instance to be used to execute the program code, the virtual machine instance having at least the first amount of the computing resource specified in the plurality of requests;

create one or more containers in the selected virtual machine instance, the one or more containers each having the first amount of the computing resource specified in the plurality of requests;

cause the program code associated with the user to be loaded from the electronic data store onto the one or more containers and executed in the one or more containers;

determine, based on a determination that the first amount of the computing resource specified in the plurality of requests has satisfied a resource adjustment condition for a threshold number of executions in the one or more containers, that the first amount should be adjusted to improve future executions of the program code;

in response to receiving additional requests specifying the program code, the user associated with the program code, and the first amount of the computing resource to be used for executing the program code, create one or more additional containers in the selected virtual machine instance such that the one or more additional containers each have a second amount of the computing resource that is different from the first amount specified in the additional requests; and

cause the program code to be executed in the one or more additional containers having the second amount of the computing resource.

8. The system of claim 7, wherein the virtual compute system is further configured to provide a notification to the user comprising one of:

(a) indicating that the first amount should be increased;

(b) indicating that the first amount should be increased by a first adjustment amount;

(c) indicating that the first amount should be decreased;

or (d) indicating that the first amount should be decreased by a second adjustment amount.

9. A system, comprising:

a virtual compute system comprising one or more hardware computing devices executing specific computer-executable instructions and configured to at least;

maintain a plurality of virtual machine instances on one or more physical computing devices;

receive a plurality of requests to execute a program code associated with a user on the virtual compute system, the request including information indicating the program code, the user associated with the program code, and a first amount of a computing resource available on the virtual compute system to be used to execute the program code;

select a virtual machine instance from the plurality of virtual machine instances to be used to execute the program code, the selected virtual machine instance having at least the first amount of the computing resource indicated in the plurality of requests;

create one or more containers in the selected virtual machine instance, the one or more containers each having the first amount of the computing resource indicated in the plurality of requests;

cause the program code associated with the user to be loaded onto the one or more containers and executed in the one or more containers;

monitor a usage of the computing resource by the program code during the executions of the program code in the one or more containers created in the selected virtual machine instance;

determine, based on the usage of the computing resource by the program code, that the first amount indicated in the plurality of requests has satisfied a resource adjustment condition for a threshold number of the executions in the one or more containers;

in response to receiving additional requests including information indicating the program code, the user associated with the program code, and the first amount of the computing resource to be used to execute the program code, create one or more additional containers in the selected virtual machine instance such that the one or more additional containers each have a second amount of the computing resource that is different from the first amount indicated in the additional requests; and

cause the program code to be executed in the one or more additional containers having the second amount of the computing resource.

10. The system of claim 9, wherein the virtual compute system is further configured to:

determine the first amount based on a user-specified amount indicated in the request or other configuration information associated with the request, the program code, or the user to execute the program code on the virtual compute system;

compare the usage of the computing resource by the program code to the user-specified amount indicated in the request; and

provide an indication of how the first amount should be adjusted.

11. The system of claim 10, wherein providing the indication comprises one of:

(a) indicating that the first amount should be increased;

(b) indicating that the first amount should be increased by a first adjustment amount;

(c) indicating that the first amount should be decreased;

or (d) indicating that the first amount should be decreased by a second adjustment amount.

12. The system of claim 9, wherein the virtual compute system is further configured to determine the second amount based on the usage of the computing resource by the program code.

13. The system of claim 12, wherein the virtual compute system is configured to determine the second amount such that the second amount is within a threshold range specified by a user associated with the program code.

14. The system of claim 9, wherein the virtual compute system is further configured to allow the usage of the computing resource by the program code to exceed the first amount by an overage amount that is less than a maximum overage amount.

15. A computer-readable, non-transitory storage medium storing computer executable instructions that, when executed by one or more computing devices, configure the one or more computing devices to:

maintain a plurality of virtual machine instances on one or more physical computing devices;

receive a plurality of requests to execute a program code associated with a user on a virtual compute system, the request including information indicating the program code, the user associated with the program code, and a first amount of a computing resource available on the virtual compute system to be used to execute the program code;

select a virtual machine instance from the plurality of virtual machine instances to be used to execute the program code, the selected virtual machine instance having at least the first amount of the computing resource indicated in the plurality of requests;

create one or more containers in the selected virtual machine instance, the one or more containers each having the first amount of the computing resource indicated in the plurality of requests;

cause the program code associated with the user to be loaded onto the one or more containers and executed in the one or more containers;

monitor a usage of the computing resource by the program code during the executions of the program code in the one or more containers created in the selected virtual machine instance;

determine, based on the usage of the computing resource by the program code, that the first amount indicated in the plurality of requests has satisfied a resource adjustment condition for a threshold number of the executions in the one or more containers;

in response to receiving additional requests including information indicating the program code, the user associated with the program code, and the first amount of the computing resource to be used to execute the program code, create one or more additional containers in the selected virtual machine instance such that the one or more additional containers each have a second amount of the computing resource that is different from the first amount indicated in the additional requests; and

cause the program code to be executed in the one or more additional containers having the second amount of the computing resource.

16. The computer-readable, non-transitory storage medium of claim 15, wherein the instructions further configure the one or more computing devices to:

determine the first amount based on a user-specified amount indicated in the request or other configuration information associated with the request, the program code, or the user to execute the program code on the virtual compute system;

compare the usage of the computing resource by the program code to the user-specified amount indicated in the request; and

provide an indication of how the first amount should be adjusted.

17. The computer-readable, non-transitory storage medium of claim 16, wherein providing the indication comprises one of:

(a) indicating that the first amount should be increased;

(b) indicating that the first amount should be increased by a first adjustment amount;

(c) indicating that the first amount should be decreased;

or (d) indicating that the first amount should be decreased by a second adjustment amount.

18. The computer-readable, non-transitory storage medium of claim 15, wherein the instructions further configure the one or more computing devices to determine the second amount based on the usage of the computing resource by the program code.

19. The computer-readable, non-transitory storage medium of claim 18, wherein determining the second amount comprises determining the second amount such that the second amount is within a threshold range specified by a user associated with the program code.

20. The computer-readable, non-transitory storage medium of claim 15, wherein the instructions further configure the one or more computing devices to allow the usage of the computing resource by the program code to exceed the first amount by an overage amount that is less than a maximum overage amount.