Optimizing a load balancer configuration

• US 10,455,009 B2
• Filed: 10/12/2018
• Issued: 10/22/2019
• Est. Priority Date: 12/18/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Moniz, Samuel Leonard, Peterson, Kyle Bradley
• Primary Examiner(s): Hoang, Hieu T
• Application Number:
  US16/159,435
• Publication Number:
  US 20190082006A1
• Time in Patent Office: 375 Days
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1025 Dynamic adaptation of the criteria on which the server selection is based
• CPC Class Codes: H04L 67/1027 Persistence of sessions during load balancing
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements

Abstract

Methods, systems, and computer-readable media for optimizing a load balancer configuration are disclosed. A load balancer is configured to perform traffic management for a distributed system. A new component is deployed to the distributed system or a performance deviation is predicted in the distributed system. In response to the initiation of the deployment or the prediction of the performance deviation, the configuration of the load balancer is modified from a first state to a second state. Modifying the configuration of the load balancer modifies the traffic management for the distributed system. If the configuration was changed due to a deployment, the configuration of the load balancer is restored from the second state to the first state after a period of time has elapsed.

Claims

1. A system, comprising:

one or more computing devices configured to implement;

in response to initiation of a deployment of one or more new components to a system, switch a load balancing configuration of a load balancer configured to direct a plurality of requests to a plurality of hosts in the system based on a load balancing configuration from a first load balancing configuration to a second load balancing configuration;

determine, based on analysis of performance metrics for the plurality of hosts that indicate an aspect of computing performance for the hosts, to switch the load balancing configuration of the load balancer; and

switch, based at least in part on said determine, the load balancing configuration of the load balancer from the second load balancing configuration to the first load balancing configuration.

2. The system as recited in claim 1, wherein, to switch the load balancing configuration of the load balancer from the first load balancing configuration to the second load balancing configuration, the one or more computing devices are configured to prioritize availability over latency and cost in the system, and wherein, to switch the load balancing configuration of the load balancer from the second balancing configuration to the first load balancing configuration, the one or more computing devices are configured to prioritize latency and cost over availability in the system.

3. The system as recited in claim 1, wherein the one or more performance metrics are used to calculate a duration of a period of time before said switch the load balancing configuration of the load balancer from the second load balancing configuration to the first load balancing configuration.

4. The system as recited in claim 1, wherein the one or more computing devices are configured to:

collect the one or more performance metrics for the system during a period of time; and

based on the one or more performance metrics, determine that the deployment is successful;

wherein the load balancing configuration of the load balancer is switched from the second load balancing configuration to the first load balancing configuration in response to said determine that the deployment is successful.

5. The system as recited in claim 1, wherein the one or more new components comprise one or more new services or one or more new versions of one or more services.

6. The system as recited in claim 5, wherein the one or more new components comprise one or more additional hosts.

7. The system as recited in claim 5, wherein the one or more computing devices are configured to:

collect, after said switch the configuration of the load balancer from the first load balancing configuration to the second load balancing configuration, one or more performance metrics for the system; and

determine, based on the one or more performance metrics, that the one or more new components have been deployed successfully;

wherein the configuration of the load balancer is restored from the second load balancing configuration to the first load balancing configuration in response to said determine that the new component has been deployed successfully.

8. A method, comprising:

collecting, for one or more hosts of a plurality of hosts of a system, one or more performance metrics that indicate an aspect of computing performance for the one or more hosts;

determining, based on the one or more performance metrics for the one or more hosts, that a future performance deviation in the system is likely;

switching, in response to said determining that the future performance deviation is likely, a load balancing configuration of a load balancer that directs a plurality of requests to the plurality of hosts in the system, from a first load balancing configuration to a second load balancing configuration; and

operating the load balancer based in the second load balancing configuration for a period of time.

9. The method as recited in claim 8, wherein said switching the configuration of the load balancer from the first load balancing configuration to the second load balancing configuration comprises switching a load balancing configuration implemented by the load balancer from a least connections configuration to a round robin configuration.

10. The method as recited in claim 8, further comprising restoring, after the period of time, the configuration of the load balancer from the second load balancing configuration to the first load balancing.

11. The method as recited in claim 10, wherein said determining, based on the one or more performance metrics for the one or more hosts, that a future performance deviation in the system is likely comprises prioritizing availability over latency and cost in the system, and wherein said restoring the configuration of the load balancer from the second load balancing configuration to the first load balancing configuration comprises prioritizing latency and cost over availability in the system.

12. The method as recited in claim 8, wherein switching the configuration of the load balancer from the first load balancing configuration to the second load balancing configuration comprises switching a respective weight for one or more hosts in a round robin load balancing configuration.

13. A non-transitory computer-readable storage medium storing program instructions computer-executable to perform:

collecting, for one or more hosts of a plurality of hosts of a system, performance metrics that indicate an aspect of computing performance for the one or more hosts;

determining, based on the one or more performance metrics for the one or more hosts, that a future performance deviation in the system is likely;

switching, in response to said determining that the future performance deviation is likely, a load balancing configuration of a load balancer that directs a plurality of requests to the plurality of hosts in the system, from a first load balancing configuration to a second load balancing configuration; and

operating the load balancer based in the second load balancing configuration for a period of time.

14. The non-transitory computer-readable storage medium as recited in claim 13, wherein the program instructions are further computer-executable to perform:

obtaining a plurality of additional performance metrics for the system after switching the load balancing configuration from the first load balancing configuration to the second load balancing configuration;

performing an analysis of the plurality of additional performance metrics; and

adjusting the load balancing configuration one or more additional times based on the analysis of the additional performance metrics.

15. The non-transitory computer-readable storage medium as recited in claim 13, wherein the program instructions are further computer-executable to perform:

collecting one or more additional performance metrics for the system during the period of time;

predicting a second performance deviation in the system based on the one or more additional performance metrics;

switching the load balancing configuration of the load balancer from the second load balancing configuration to a third load balancing configuration in response to predicting the second performance deviation; and

operating the load balancer based on the third load balancing configuration for a second period of time.

16. The non-transitory computer-readable storage medium as recited in claim 13, wherein the first load balancing configuration comprises a first set of weights for the plurality of hosts in a round robin load balancing algorithm, and wherein the second load balancing configuration comprises a second set of weights for the plurality of hosts in the round robin load balancing algorithm.

17. The non-transitory computer-readable storage medium as recited in claim 13, wherein the load balancing configuration comprises a load balancing algorithm, wherein the first load balancing configuration comprises a first load balancing algorithm, and wherein the second load balancing configuration comprises a second load balancing algorithm.

18. The non-transitory computer-readable storage medium as recited in claim 13, wherein a duration of the period of time is determined based on one or more additional performance metrics.

19. The non-transitory computer-readable storage medium as recited in claim 13, wherein a duration of the period of time is determined based on user input.

20. The non-transitory computer-readable storage medium as recited in claim 13, wherein said determining that the future performance deviation in the system is likely comprises determining one or more indicators of an impending performance degradation relative to a threshold.