Providing dynamically scaling computing load balancing

• US 9,864,727 B1
• Filed: 09/15/2012
• Issued: 01/09/2018
• Est. Priority Date: 05/28/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Miller, Kevin Christopher, Brandwine, Eric Jason
• Primary Examiner(s): Nguyen, Minh-Chau
• Application Number:
  US13/620,806
• Time in Patent Office: 1,942 Days
• Field of Search: 709221-224
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: H04L 67/1004 Server selection for load balancing
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

Techniques are described for providing load balancing functionality among multiple computing nodes. In some situations, the provided load balancing functionality includes dynamically scaling a group of multiple computing nodes for which the load balancing is performed, such as to dynamically expand and/or shrink the quantity of computing nodes in the group based on predefined criteria. At least some of the computing nodes of a group may be part of one or more physical computer networks in one or more geographical locations under control of a user or other entity, and at least some of the dynamic scaling of the group may use one or more other computing nodes that are part of a remote computer network (e.g., a virtual computer network provided under the control of a network-accessible service). The defined criteria used for the dynamic scaling may be determined in various manners and based on various factors.

Claims

1. A computer-implemented method comprising:

receiving, by one or more computing devices of a network service, instructions from a client that include one or more criteria for using provided computing nodes to supplement functionality supplied by a group of computing nodes in a first computer network of the client that is external to the network service;

responding, by the one or more computing devices, to a first request for functionality from the group of computing nodes in the first computer network by selecting at least one of the computing nodes of the group to handle the first request, and initiating handling of the first request by the selected at least one computing node;

determining, by the one or more computing devices, that the one or more criteria for the group are satisfied;

based on the determining, initiating, by the one or more computing devices, a modification to the group of computing nodes in the first computer network by implementing one or more additional computing nodes provided by the network service that also provide the functionality of the group; and

responding, by the one or more computing devices and after the modification, to a second request for the functionality by selecting at least one of the additional computing nodes to handle the second request, and initiating handling of the second request by the selected at least one additional computing node.

2. The computer-implemented method of claim 1 wherein the initiating of the modification to the group further includes provisioning the one or more additional computing nodes by executing one or more software applications on the provisioned computing nodes.

3. The computer-implemented method of claim 2 further comprising determining the one or more software applications to execute on the one or more additional computing nodes based at least in part on a quantity of other computing nodes of the group executing a particular software application, on a geographical or network location of stored data for the one or more software applications, or on a geographical or network location of one or more computing resources to be used by the one or more software applications.

4. The computer-implemented method of claim 1 wherein the network service is a configurable network service, and wherein the method further comprises implementing, by the one or more computing systems and based at least in part on the instructions from the client, a second computer network that is a virtual computer network provided for use of the client by the configurable network service, and wherein the one or more additional computing nodes are added to the second computer network.

5. The computer-implemented method of claim 4 wherein the first computer network is at a first geographical location, and wherein the virtual computer network is at a remote second geographical location.

6. The computer-implemented method of claim 4 wherein the first computer network is at a first geographical location, and wherein the virtual computer network includes computing nodes at multiple geographical locations distinct from the first geographical location.

7. The computer-implemented method of claim 4 further comprising receiving the one or more criteria via a programmatic interface provided by the configurable network service for use by the client.

8. The computer-implemented method of claim 4 wherein the initiating of the modification to the group of computing nodes includes using a programmatic interface of the configurable network service to add the one or more additional computing nodes of the second computer network to the group.

9. The computer-implemented method of claim 4 wherein the first computer network is at a first geographical location, wherein the virtual computer network is at a remote second geographical location, and wherein the one or more computing devices are at the remote second location and are provided by the configurable network service for use in providing load balancing capabilities for the group.

10. The computer-implemented method of claim 4 wherein the one or more additional computing nodes are each a virtual machine hosted on one of multiple physical computing systems of the configurable network service.

11. The computer-implemented method of claim 1 wherein each of the computing nodes of the group before the modification executes a copy of one or more software applications to provide the functionality, and wherein the initiating of the modification to the group further includes executing a copy of the one or more software applications on each of the one or more additional computing nodes.

12. The computer-implemented method of claim 1 wherein the one or more criteria for the group are based at least in part on predicted demand for use of computing resources by the computing nodes of the group.

13. The computer-implemented method of claim 1 wherein the one or more criteria include a computing load of at least one of the computing nodes of the group exceeding one or more defined thresholds, and wherein the determining that the one or more criteria for the group are satisfied includes monitoring the computing load on at least one computing node of the group and using the monitored computing load for the determining.

14. The computer-implemented method of claim 1 wherein the one or more criteria for the group are based at least in part on status information indicating a volume of requests being received by at least one of the computing nodes of the group, a degree of network utilization of at least one of the computing nodes of the group, a degree of network latency for at least one of the computing nodes of the group, a degree of latency in responding to requests for at least one of the computing nodes of the group, a degree of processor utilization for at least one of the computing nodes of the group, a degree of memory utilization for at least one of the computing nodes of the group, or a degree of bandwidth utilization for at least one of the computing nodes of the group, and wherein the determining that the one or more criteria for the group are satisfied includes obtaining the status information and using the obtained status information for the determining.

15. The computer-implemented method of claim 1 wherein the one or more criteria for the group are based at least in part on a first cost of using at least one of the multiple computing nodes of the group and on a second cost of using the one or more additional computing nodes, and wherein the determining that the one or more criteria for the group are satisfied includes obtaining information about the first and second costs and using the obtained information for the determining.

16. The computer-implemented method of claim 1 further comprising, after the initiating of the modification to the group of computing nodes, determining that one or more other criteria for the group are satisfied, and initiating a further modification to the group that includes removing from the group one or more computing nodes that are provided by the network service and are part of the second computer network.

17. A non-transitory computer-readable medium having stored contents that cause one or more computing devices of a network service to:

receive, by the one or more computing devices of the network service, instructions from a client regarding using provided computing nodes of the network service to improve functionality supplied by a group of one or more computing nodes in a first computer network of the client that is external to the network service;

respond, by the one or more computing devices, to a first request for the functionality by selecting at least one of the computing nodes of the group to respond to the first request;

determine, by the one or more computing devices, that one or more defined criteria for the group are satisfied;

initiate, by the one or more computing devices, one or more modifications to the group of computing nodes that include adding one or more additional computing nodes to the group from a distinct virtual computer network provided by the network service; and

respond, by the one or more computing devices, to a second request for the functionality by selecting at least one of the additional computing nodes to respond to the second request.

18. The non-transitory computer-readable medium of claim 17 wherein each of the one or more computing nodes of the group before the one or more modifications execute a copy of one or more software applications to provide the functionality, and wherein the initiating of the one or more modifications to the group further includes executing a copy of the one or more software applications on each of the one or more additional computing nodes.

19. The non-transitory computer-readable medium of claim 17 wherein the stored contents further cause the one or more computing devices to provide one or more interfaces for use by clients of the network service and receive the defined criteria for the group via at least one of the provided interfaces.

20. The non-transitory computer-readable medium of claim 17 wherein the determining that the one or more defined criteria are satisfied includes receiving status information indicating that at least one of the one or more computing nodes of the group has exceeded one or more defined thresholds, and using the received status information for the determining.

21. The non-transitory computer-readable medium of claim 17 wherein the first computer network is at a first geographical location, and wherein the virtual computer network includes computing nodes at one or more other geographical locations that are distinct from the first geographical location.

22. The non-transitory computer-readable medium of claim 17 wherein the stored contents further cause the one or more computing devices to determine that one or more other defined criteria for the group are satisfied, and initiate one or more other modifications to the group of computing nodes that include removing from the group one or more computing nodes of the virtual computer network.

23. A computing device comprising:

one or more processors; and

one or more memories having stored instructions of a network service that, when executed by at least one of the one or more computer processors, cause the computing device to at least;

receive instructions from a client regarding using provided computing nodes of the network service to improve functionality supplied by a group of one or more first computing nodes of the client at one or more first geographical locations external to the network service;

respond to one or more first requests for the functionality by selecting at least one of the first computing nodes to handle the one or more first requests;

determine that one or more defined criteria for the group are satisfied, and responsive to the determining, initiate a modification to the group to add one or more second computing nodes from the network service at a distinct second location to the group; and

respond to one or more second requests for the functionality by selecting one of the one or more second computing nodes to handle the one or more second requests.

24. The computing device of claim 23 wherein the determining that the one or more defined criteria are satisfied receiving information that at least one computing node of the group has exceeded one or more defined thresholds, and using the received information for the determining.

25. The computing device of claim 23 wherein the stored instructions further cause the computing device to implement a load balancer in the network service that performs the responding to the one or more first requests and the one or more second requests and that is configured to determine that one or more other defined criteria for the group are satisfied, and initiate one or more other modifications to the group that include removing from the group one or more computing nodes provided by the network service.