Dynamically modifying program execution capacity

• US 10,642,653 B2
• Filed: 03/20/2015
• Issued: 05/05/2020
• Est. Priority Date: 09/29/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Maclinovsky, Alex, Meike, Blake, Buragohain, Chiranjeeb, Kommareddy, Christopher Reddy, Pare, Geoffrey Scott, Heitmann, John W., Lohia, Sumit, Chen, Liang, Musgrave, Zachary S.
• Primary Examiner(s): Lazaro, David R
• Application Number:
  US14/664,592
• Publication Number:
  US 20150193276A1
• Time in Patent Office: 1,873 Days
• Field of Search: 709201-207, 709212-215, 709217-222, 709223-226, 709227, 709229, 709230, 709232-235, 709249
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 30/04 Billing or invoicing
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: Y04S 10/50 Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

Techniques are described for managing program execution capacity, such as for a group of computing nodes that are provided for executing one or more programs for a user. In some situations, dynamic program execution capacity modifications for a computing node group that is in use may be performed periodically or otherwise in a recurrent manner, such as to aggregate multiple modifications that are requested or otherwise determined to be made during a period of time, and with the aggregation of multiple determined modifications being able to be performed in various manners. Modifications may be requested or otherwise determined in various manners, including based on dynamic instructions specified by the user, and on satisfaction of triggers that are previously defined by the user. In some situations, the techniques are used in conjunction with a fee-based program execution service that executes multiple programs on behalf of multiple users of the service.

Claims

1. A computer-implemented method comprising:

obtaining, by a configured computing system of a program execution service including a plurality of computing resources available to users, instructions from a first user of the program execution service for use in later modifying an amount of allocated computing resources in a group provided for the first user by the program execution service from the plurality of computing resources, wherein the obtained instructions from the first user specify a threshold for CPU (central processing unit) utilization of the allocated computing resources in the group during executing of one or more programs for the first user and further specify to add to or remove from the group a quantity of multiple computing nodes if the specified threshold is satisfied, wherein the quantity of multiple computing nodes is specified by the first user to be determined using a defined proportion of an existing quantity of allocated computing nodes in the group;

monitoring, by the program execution service, the CPU utilization of the allocated computing resources in the group during the executing of the one or more programs;

determining, by the program execution service and based at least in part on the monitoring indicating that the CPU utilization of the allocated computing resources in the group satisfies the threshold specified by the first user, to modify the allocated computing resources in the group, including to determine the quantity of multiple computing nodes for the modifying by using the defined proportion of the existing quantity of allocated computing nodes in the group as specified by the first user; and

initiating, by the program execution service and based at least in part on the determining, modifying an amount of the allocated computing resources in the group by the determined quantity of multiple computing nodes.

2. The computer-implemented method of claim 1 further comprising providing, by the program execution service and before the executing of the one or more programs, the group of allocated computing resources for use on behalf of the first user, including provisioning one or more computing nodes of the allocated computing resources for use in performing the executing of the one or more programs.

3. The computer-implemented method of claim 2 wherein the obtained instructions from the first user further specify an initial amount of computing resources for the group, and wherein the method further comprises selecting, by the program execution service and for use in the group, the initial amount of computing resources from the computing resources available to the users.

4. The computer-implemented method of claim 2 further comprising executing, by the program execution service and on behalf of the first user, the one or more programs on the provisioned one or more computing nodes.

5. The computer-implemented method of claim 1 further comprising determining that the CPU utilization reaches the specified threshold, the specified threshold representing a maximum threshold, and performing the modifying of the amount of the allocated computing resources in the group by adding the determined quantity of multiple computing nodes to the group.

6. The computer-implemented method of claim 1 further comprising determining that the CPU utilization reaches the specified threshold, the specified threshold representing a minimum threshold, and performing the modifying of the amount of the allocated computing resources in the group by removing the determined quantity of multiple computing nodes from the group.

7. The computer-implemented method of claim 1 wherein the determining to modify the allocated computing resources is performed while the allocated computing resources of the group are in use for the executing of the one or more programs for the first user.

8. The computer-implemented method of claim 1 wherein the modifying of the amount of the allocated computing resources in the group is performed while the allocated computing resources of the group are in use for the executing of the one or more programs for the first user.

9. The computer-implemented method of claim 1 wherein the obtaining of the instructions further includes obtaining information from the first user to specify a capacity modification trigger that includes one or more criteria for use in determining if the capacity modification trigger is satisfied, and wherein the monitoring by the program execution service includes further monitoring the one or more criteria.

10. The computer-implemented method of claim 9 wherein the capacity modification trigger further indicates the defined proportion, and wherein the method further comprises determining, by the program execution service, that the specified capacity modification trigger is satisfied based on the further monitoring.

11. The computer-implemented method of claim 1 further comprising, by the program execution service, obtaining a capacity modification instruction from the first user that indicates a first amount to modify the allocated computing resources, and performing the modifying based at least in part on the first amount indicated in the capacity modification instruction.

12. The computer-implemented method of claim 1 wherein the program execution service provides one or more application programming interfaces (APIs) for use in programmatic interactions with the program execution service over one or more networks, and wherein the obtaining of the instructions from the first user includes obtaining the instructions from an executing program that performs programmatic interactions over the one or more networks with the program execution service on behalf of the first user.

13. The computer-implemented method of claim 1 wherein the program execution service provides virtualized computing resources that are available to the users, and wherein the allocated computing nodes of the group include multiple virtual machines provided using physical computing machines of the program execution service.

14. The computer-implemented method of claim 1 wherein the obtained instructions further indicate a second threshold that is specified by the first user and is associated with an ability of the computing nodes to respond in a load-balanced manner, and wherein the modifying of the amount of the allocated computing resources includes adding one or more additional computing nodes to the group based at least in part on the second threshold being determined to be satisfied based on the monitoring.

15. The computer-implemented method of claim 1 wherein the defined proportion is one of a first relative amount with respect to the existing amount of allocated computing nodes in the group and a second relative amount with respect to the existing amount of allocated computing nodes in the group, and wherein determining of the quantity of multiple computing nodes for the modifying includes selecting the first relative amount or the second relative amount of modifying to perform based on a determined level of CPU utilization, wherein the first relative amount is selected if the determined level of CPU utilization exceeds a first threshold and the second relative amount is selected if the determined level of CPU utilization exceeds a second threshold, and wherein the initiating of the modifying of the amount of the allocated computing resources in the group includes performing the modifying using the selected first relative amount or second relative amount.

16. The computer-implemented method of claim 1 wherein the specified threshold for CPU utilization is for an average CPU utilization by the allocated computing resources in the group over a period of time, and wherein the monitoring includes determining the average CPU utilization by the allocated computing resources in the group over the period of time.

17. A non-transitory computer-readable medium including stored contents that cause a computing system of a program execution service to:

obtain, by the computing system, instructions from a client of the program execution service for modifying allocated computing resources of a group of computing nodes provided by the program execution service on behalf of the client if a threshold for CPU (central processing unit) utilization is satisfied for the computing nodes of the group, wherein the obtained instructions specify a quantity of multiple computing nodes to add to or remove from the group if the threshold is satisfied;

monitor, by the computing system, the CPU utilization of the computing nodes of the group;

determine, by the computing system and based at least in part on the monitoring indicating that the CPU utilization of the computing nodes of the group satisfies the threshold, to modify the allocated computing resources of the group by the quantity of multiple computing nodes specified by the client; and

initiate, by the computing system, modifying an amount of the allocated computing resources of the group by the quantity of multiple computing nodes specified by the client.

18. The non-transitory computer-readable medium of claim 17 wherein the obtained instructions further include a request from the client to provide an initial amount of computing resources from the group of computing nodes, and wherein the stored contents include software instructions that, when executed, further cause the computing system to allocate the initial amount of computing resources and to initiate execution of one or more software programs on behalf of the client.

19. The non-transitory computer-readable medium of claim 18 wherein the software instructions further cause the computing system to perform the modifying of the amount of the allocated computing resources by adding the quantity of multiple computing nodes to the group from a plurality of computing nodes provided by the program execution service for use by users of the program execution service.

20. The non-transitory computer-readable medium of claim 19 wherein the determining to modify the allocated computing resources is performed while the one or more software programs are being executed by the computing nodes of the group, and wherein the initiating of the modifying of the amount of allocated computing resources is performed in response to the determining.

21. A system comprising:

one or more hardware processors; and

one or more memories with stored instructions that, when executed by at least one of the hardware processors, cause the system to;

provide, for a first user of a program execution service, an initial amount of computing resources using a group of computing nodes provided by the program execution service;

obtain instructions from the first user for use in later modifying an amount of computing resources of the group of computing nodes if a specified threshold is satisfied for CPU (central processing unit) utilization of the computing nodes of the group, wherein the obtained instructions specify a quantity of multiple computing nodes to add to or remove from the group if the specified threshold is satisfied;

monitor the CPU utilization of the computing nodes of the group;

determine, based at least in part on the monitoring indicating that the CPU utilization of the computing nodes of the group satisfies the specified threshold, to modify the amount of computing resources of the group of computing nodes by the specified quantity of multiple computing nodes; and

initiate modifying of the amount of computing resources of the group of computing nodes by the specified quantity of multiple computing nodes.

22. The system of claim 21 wherein the stored instructions are part of the program execution service and further configure the at least one hardware processor to perform the modifying of the amount of computing resources of the group of computing nodes by adding, to the group, the specified quantity of multiple computing nodes from a plurality of computing nodes that are provided by the program execution service for use by its users.

23. The system of claim 22 wherein the monitoring and the determining are performed while one or more software programs are being executed by the computing nodes of the group on behalf of the first user, and wherein the initiating of the modifying of the amount of computing resources of the group of computing nodes is performed while the one or more software programs continue to be executed by the computing nodes of the group.

24. A computer-implemented method comprising:

obtaining, by a configured computing system of a program execution service including a plurality of computing resources available to users, instructions from a first user of the program execution service for modifying an amount of allocated computing resources in a group provided for the first user by the program execution service from the plurality of computing resources, wherein the obtained instructions from the first user specify a threshold for network bandwidth utilization by the allocated computing resources in the group during executing of one or more programs for the first user and further specify a quantity of multiple computing nodes to add to or remove from the group if the specified threshold is satisfied;

monitoring, by the program execution service, the network bandwidth utilization by the allocated computing resources in the group during the executing of the one or more programs;

determining, by the program execution service and based at least in part on the monitoring indicating that the network bandwidth utilization by the allocated computing resources in the group satisfies the threshold specified by the first user, to modify the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user; and

initiating, by the program execution service and based at least in part on the determining, modifying an amount of the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user.

25. The computer-implemented method of claim 24 wherein the specified threshold for network bandwidth utilization is for an average network bandwidth utilization by the allocated computing resources in the group over a period of time, wherein the monitoring includes determining the average network bandwidth utilization by the allocated computing resources in the group over the period of time, and wherein the modifying of the amount of the allocated computing resources in the group is performed based at least in part on the determined average network bandwidth utilization exceeding the specified threshold for the average network bandwidth utilization and while the allocated computing resources of the group are in use for the executing of the one or more programs for the first user.

26. A computer-implemented method comprising:

obtaining, by a configured computing system of a program execution service including a plurality of computing resources available to users, instructions from a first user of the program execution service for modifying an amount of allocated computing resources in a group provided for the first user by the program execution service from the plurality of computing resources, wherein the obtained instructions from the first user specify a threshold for memory utilization by the allocated computing resources in the group during executing of one or more programs for the first user and further specify a quantity of multiple computing nodes to add to or remove from the group if the specified threshold is satisfied;

monitoring, by the program execution service, the memory utilization by the allocated computing resources in the group during the executing of the one or more programs;

determining, by the program execution service and based at least in part on the monitoring indicating that the memory utilization by the allocated computing resources in the group satisfies the threshold specified by the first user, to modify the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user; and

initiating, by the program execution service and based at least in part on the determining, modifying an amount of the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user.

27. The computer-implemented method of claim 26 wherein the specified threshold for memory utilization is for an average memory utilization by the allocated computing resources in the group over a period of time, wherein the monitoring includes determining the average memory utilization by the allocated computing resources in the group over the period of time, and wherein the modifying of the amount of the allocated computing resources in the group is performed based at least in part on the determined average memory utilization exceeding the specified threshold for the average memory utilization and while the allocated computing resources of the group are in use for the executing of the one or more programs for the first user.

28. A computer-implemented method comprising:

obtaining, by a configured computing system of a program execution service including a plurality of computing resources available to users, instructions from a first user of the program execution service for modifying an amount of allocated computing resources in a group provided for the first user by the program execution service from the plurality of computing resources, wherein the obtained instructions from the first user specify a threshold for storage utilization by the allocated computing resources in the group during executing of one or more programs for the first user and further specify a quantity of multiple computing nodes to add to or remove from the group if the specified threshold is satisfied;

monitoring, by the program execution service, the storage utilization by the allocated computing resources in the group during the executing of the one or more programs;

determining, by the program execution service and based at least in part on the monitoring indicating that the storage utilization by the allocated computing resources in the group satisfies the threshold specified by the first user, to modify the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user; and

initiating, by the program execution service and based at least in part on the determining, modifying an amount of the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user.

29. The computer-implemented method of claim 28 wherein the specified threshold for storage utilization is for an average storage utilization by the allocated computing resources in the group over a period of time, wherein the monitoring includes determining the average storage utilization by the allocated computing resources in the group over the period of time, and wherein the modifying of the amount of the allocated computing resources in the group is performed based at least in part on the determined average storage utilization exceeding the specified threshold for the average storage utilization and while the allocated computing resources of the group are in use for the executing of the one or more programs for the first user.

30. A computer-implemented method comprising:

obtaining, by a configured computing system of a program execution service including a plurality of computing resources available to users, instructions from a first user of the program execution service for modifying an amount of allocated computing resources in a group provided for the first user by the program execution service from the plurality of computing resources, wherein the obtained instructions from the first user specify a threshold for utilization, with respect to one or more indicated metrics, by the allocated computing resources in the group during executing of one or more programs for the first user;

monitoring, by the program execution service, the utilization by the allocated computing resources in the group with respect to the one or more indicated metrics during the executing of the one or more programs;

determining, by the program execution service and based at least in part on the monitoring indicating that the utilization by the allocated computing resources in the group with respect to the one or more indicated metrics satisfies the threshold specified by the first user, to modify the allocated computing resources in the group; and

initiating, by the program execution service and based at least in part on the determining, modifying an amount of the allocated computing resources in the group.

31. The computer-implemented method of claim 30 wherein the specified threshold is for an average utilization by the allocated computing resources in the group over a period of time with respect to the one or more indicated metrics, wherein the monitoring includes determining the average utilization by the allocated computing resources in the group over the period of time with respect to the one or more indicated metrics, wherein the obtained instructions from the first user for modifying the amount of allocated computing resources in the group further specify a quantity of multiple computing nodes to add to or remove from the group if the specified threshold is satisfied, and wherein the modifying of the amount of the allocated computing resources in the group includes modifying the amount of the allocated computing resources in the group by the quantity of multiple computing nodes specified by the first user and is performed based at least in part on the determined average utilization exceeding the specified threshold for the average utilization and while the allocated computing resources of the group are in use for the executing of the one or more programs for the first user.