Automatic scaling of resource instance groups within compute clusters

• US 10,581,964 B2
• Filed: 12/18/2017
• Issued: 03/03/2020
• Est. Priority Date: 05/01/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Einkauf, Jonathan Daly, Natali, Luca, Kalathuru, Bhargava Ram, Baji, Saurabh Dileep, Sinha, Abhishek Rajnikant
• Primary Examiner(s): Higa, Brendan Y
• Application Number:
  US15/845,855
• Publication Number:
  US 20180109610A1
• Time in Patent Office: 806 Days
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/0897 by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 41/5045 Making service definitions prior to deployment
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
• CPC Class Codes: H04L 67/1076 Resource dissemination mechanisms or network resource keeping policies for optimal resource availability in the overlay network

Abstract

A service provider may apply customer-selected or customer-defined auto-scaling policies to a cluster of resources (e.g., virtualized computing resource instances or storage resource instances in a MapReduce cluster). Different policies may be applied to different subsets of cluster resources (e.g., different instance groups containing nodes of different types or having different roles). Each policy may define an expression to be evaluated during execution of a distributed application, a scaling action to take if the expression evaluates true, and an amount by which capacity should be increased or decreased. The expression may be dependent on metrics emitted by the application, cluster, or resource instances by default, metrics defined by the client and emitted by the application, or metrics created through aggregation. Metric collection, aggregation and rules evaluation may be performed by a separate service or by cluster components. An API may support auto-scaling policy definition.

Claims

1. A method, comprising:

performing, by one or more computers;

detecting that a trigger condition has been met during execution of a distributed application on a cluster of computing resource instances, wherein the cluster comprises two or more non-overlapping instance groups and each instance group comprises a respective one or more computing resource instances; and

in response to said detecting, performing an automatic scaling operation on a particular instance group of the non-overlapping instance groups, wherein the particular instance group on which to perform the automatic scaling operation is determined prior to detecting the trigger condition, wherein determination of the particular instance group is based at least in part on input received from a client of a distributed computing system that includes the cluster, wherein the automatic scaling operation changes the number of computing resource instances on the particular instance group without changing the number of computing resource instances on at least another one of the two or more instance groups.

2. The method of claim 1,

wherein the trigger condition comprises an expression that, when evaluated true, triggers the performance of the automatic scaling operation on the one of the instance groups, and wherein the expression is dependent on one or more metrics generated during execution of the distributed application on the cluster.

3. The method of claim 1,

wherein the trigger condition comprises an expression that, when evaluated true, triggers the performance of the automatic scaling operation on the one of the instance groups, and wherein the expression is dependent on a day of the week, a date, a time of day, an elapsed period of time, or an estimated period of time.

4. The method of claim 1, further comprising:

detecting that another trigger condition has been met during execution of the distributed application on the cluster; and

in response to detecting that the other trigger condition has been met, initiating performance of another automatic scaling operation that changes the number of compute resource instances in another one of the plurality of instance groups.

5. The method of claim 1,

wherein the automatic scaling operation comprises an operation to add capacity to the one instance group.

6. The method of claim 1,

wherein the automatic scaling operation comprises an operation to remove capacity from the one instance group.

7. The method of claim 1, further comprising:

receiving, by the cluster, an automatic scaling policy that defines an amount by which the automatic scaling operation changes a capacity of the one instance group or a percentage by which the automatic scaling operation changes a capacity of the one instance group.

8. A distributed computation system, comprising:

one or more computers that comprise at least a processor and a memory and that implement a cluster that comprises two or more non-overlapping instance groups of one or more computing resource instances,

wherein the distributed computation system is to;

detect that a trigger condition has been met during execution of a distributed application on the cluster of computing resource instances; and

in response to detection that the trigger condition has been met, perform an automatic scaling operation on a particular instance group of the non-overlapping instance groups, wherein the particular instance group on which to perform the automatic scaling operation is determined prior to detecting the trigger condition, wherein determination of the particular instance group is based at least in part on input received from a client of a distributed computing system that includes the cluster, wherein the automatic scaling operation changes the number of computing resource instances on the particular instance group without changing the number of computing resource instances on at least another one of the two or more instance groups.

9. The system of claim 8,

wherein the distributed application is to emit one or more application-specific metrics;

and

wherein the trigger condition is dependent at least in part on at least one of the one or more application-specific metrics.

10. The system of claim 8,

wherein the distributed computation system is to:

receive one or more metrics from a respective monitor component on each of at least two of the computing resource instances; and

aggregate the metrics received from the respective monitor components to generate an aggregate metric for the at least two of the computing resource instances; and

wherein the trigger condition is determined based at least in part on the aggregate metric.

11. The system of claim 8, wherein the trigger condition comprises an expression that, when evaluated true, triggers the performance of the automatic scaling operation, and wherein the expression is dependent on a day of the week, a date, a time of day, an elapsed period of time, or an estimated period of time.

12. The system of claim 8, further comprising an interface to receive one or more inputs that define an automatic scaling policy that determines an amount by which the automatic scaling operation is to change the number of nodes of the one instance group or a percentage by which the automatic scaling operation is to change the number of nodes of the one instance group.

13. The system of claim 8, wherein the automatic scaling operation comprises an operation to add capacity to the one instance group.

14. The system of claim 8, wherein the automatic scaling operation comprises an operation to remove capacity from the one instance group.

15. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more computers cause the one or more computers to:

detect that a trigger condition has been met during execution of a distributed application on a cluster of computing resource instances, wherein the cluster comprises two or more non-overlapping instance groups and each instance group comprises a respective one or more computing resource instances; and

in response to said detection, perform an automatic scaling operation on a particular instance group of the instance groups, wherein the particular instance group on which to perform the automatic scaling operation is determined prior to detecting the trigger condition, wherein determination of the particular instance group is based at least in part on input received from a client of a distributed computing system that includes the cluster, wherein the automatic scaling operation changes the number of computing resource instances on the particular instance group without changing the number of computing resource instances on at least another one of the two or more instance groups.

16. The non-transitory computer-accessible storage medium of claim 15, wherein the program instructions when executed on one or more computers further cause the one or more computers to receive, through an interface from a client, input that comprises information that defines an expression that, when evaluated true, determines that the trigger condition has been met to perform the automatic scaling operation.

17. The non-transitory computer-accessible storage medium of claim 16, where the expression is dependent at least in part on one or more of:

a day of the week, a date, a time of day, an elapsed period of time, an estimated period of time, a resource utilization metric, a cost metric, an estimated time to complete execution of a task on behalf of the distributed application, or a number of pending tasks to be performed on behalf of the distributed application.

18. The non-transitory computer-accessible storage medium of claim 15,

wherein the distributed application is to emit one or more application-specific metrics;

and

wherein the trigger condition is dependent at least in part on at least one of the one or more application-specific metrics.

19. The non-transitory computer-accessible storage medium of claim 15, wherein the automatic scaling operation comprises an operation to add capacity to the one of the two or more instance groups.

20. The non-transitory computer-accessible storage medium of claim 15, wherein the automatic scaling operation comprises an operation to remove capacity from the one of the two or more instance groups.