Providing dynamically scaling computing load balancing

• US 8,296,434 B1
• Filed: 05/28/2009
• Issued: 10/23/2012
• Est. Priority Date: 05/28/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Miller, Kevin Christopher, Brandwine, Eric Jason
• Primary Examiner(s): Nguyen, Minh-Chau
• Application Number:
  US12/474,145
• Time in Patent Office: 1,244 Days
• Field of Search: 709/226, 709/220, 709/224
• US Class Current: 709/226
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: H04L 67/1004 Server selection for load balancing
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer

Abstract

Techniques are described for providing load balancing functionality among multiple computing nodes. In some situations, the provided load balancing functionality includes dynamically scaling a group of multiple computing nodes for which the load balancing is performed, such as to dynamically expand and/or shrink the quantity of computing nodes in the group based on predefined criteria. At least some of the computing nodes of a group may be part of one or more physical computer networks in one or more geographical locations under control of a user or other entity, and at least some of the dynamic scaling of the group may use one or more other computing nodes that are part of a remote computer network (e.g., a virtual computer network provided under the control of a network-accessible service). The defined criteria used for the dynamic scaling may be determined in various manners and based on various factors.

Claims

1. A non-transitory computer-readable medium whose contents are instructions that, when executed, configure a device to provide load balancing capabilities by performing a method comprising:

for each of one or more requests received for functionality that is available from each of multiple computing nodes of a group at a first location, selecting at least one of the computing nodes of the group to handle the request;

determining that one or more defined criteria for the group are satisfied, and initiating a modification to the group of computing nodes, the modification to the group including modifying use as part of the group of one or more additional computing nodes at a second location remote from the first location; and

for each of one or more additional requests for the available functionality, selecting at least one of the computing nodes of the modified group to handle the additional request, such that at least one additional computing node at the remote second location is selected for at least one of the additional requests.

2. The non-transitory computer-readable medium of claim 1 wherein the multiple computing nodes of the group are associated with a first entity that specifies the defined criteria for the group, wherein the defined criteria are based at least in part on status information obtained about one or more of the computing nodes that are part of the group before the initiating of the modification to the group, wherein the computing nodes of the group each execute a copy of one or more software applications to provide the available functionality, and wherein the one or more requests and the one or more additional requests are each from one or more computing systems that are not part of the group to obtain the available functionality from the one or more software applications.

3. The non-transitory computer-readable medium of claim 2 wherein the modifying of the use of the one or more additional computing nodes at the remote second location includes at least one of adding the one or more additional computing nodes to the group and of removing the one or more additional computing nodes from the group.

4. The non-transitory computer-readable medium of claim 3 wherein the multiple computing nodes that are part of the group before the initiating of the modification are part of a computer network at the first location and are under control of the first entity, wherein the one or more additional computing nodes at the remote second location are part of a first virtual computer network at the remote second location and are under control of a distinct second entity, wherein the second entity provides a network-accessible service that enables clients to configure virtual computer networks provided by the network-accessible service on behalf of the clients, wherein the first entity is one of the clients of the network-accessible service, and wherein the first virtual computer network is one of the virtual computer networks provided by the network-accessible service.

5. The non-transitory computer-readable medium of claim 1 wherein the computer-readable medium is a memory of the configured device that stores the contents.

26. The method of claim 1 further comprising, under control of one or more computing systems that provide the configurable network service, and for the load balancer device:

receiving one or more requests from the load balancer device via the programmatic interface of the configurable network service, the received one or more requests indicating the configurable virtual computer network that is an extension to the physical network of the first user, the indicated virtual computer network being one of multiple virtual computer networks provided by the configurable network service to multiple remote clients, the first user being one of the multiple remote clients, and indicating one or more operations to be performed with respect to the indicated virtual computer network on behalf of the first user, the indicated one or more operations including causing one or more computing nodes to be added to the indicated virtual computer network and including provisioning the one or more added computing nodes of the indicated virtual computer network to execute the software application for use in the providing of the common type of functionality; and

in response to the one or more requests,

automatically selecting one or more of a plurality of computing nodes provided by the configurable network service that are not currently part of any of the multiple virtual computer networks, and assigning the selected computing nodes to the indicated virtual computer network, such that the first user has access to the assigned computing nodes via the indicated virtual computer network; and

provisioning the selected one or more computing nodes to execute the software application, such that the provisioned one or more computing nodes of the indicated virtual computer network are ready to receive and handle forwarded requests for the provided common type of functionality from other computing systems.

27. The method of claim 1 wherein the configurable network service is a fee-based service that is accessible to remote clients via public networks and that provides multiple virtual computer networks using cloud computing techniques, and wherein the first user is one of the remote clients and provides payment to the configurable network service for the interacting with the programmatic interface of the configurable network service to cause the one or more additional computing nodes to be added to the configurable virtual computer network provided to the first user.

6. A computer-implemented method for providing load balancing capabilities, the method comprising:

under control of one or more computing devices providing load balancing capabilities for a group of multiple computing nodes providing a type of network-accessible functionality, the multiple computing nodes belonging to one or more computer networks,

after receiving one or more requests from one or more computing systems for the provided type of functionality, selecting for each of the received requests one of the computing nodes of the group to respond to the request;

after the receiving of the one or more requests, automatically determining that one or more defined criteria for the group are satisfied, the one or more defined criteria being based at least in part on status information for one or more of the computing nodes of the group;

in response to the determining that the one or more defined criteria are satisfied, automatically initiating a modification to the group of computing nodes, the modification to the group including adding one or more additional computing nodes to the group that are part of a remote virtual computer network distinct from the one or more computer networks; and

after receiving one or more additional requests from one or more computing systems for the provided type of functionality, selecting for each of the received additional requests one of the computing nodes of the modified group to respond to the additional request, such that at least one of the additional computing nodes is selected for at least one of the additional requests.

7. The method of claim 6 wherein the computing nodes that are part of the group before the initiating of the modification to the group each execute a copy of one or more software applications to provide the type of network-accessible functionality, and wherein the initiating of the modification to the group further includes initiating provisioning of the added one or more additional computing nodes, the provisioning including executing a copy of the one or more software applications on each of the added one or more additional computing nodes.

8. The method of claim 7 wherein the initiating of the modification to the group further includes initiating providing one or more logical storage volumes for use by the executing software application copies on the added one or more additional computing nodes.

9. The method of claim 6 wherein the one or more defined criteria for the group are further based in part on a cost of using at least one of the one or more computing nodes of the group and of the added one or more additional computing nodes.

10. The method of claim 9 wherein the cost of using the at least one computing nodes of the group and of the added one or more additional computing nodes is at least one of a monetary cost and an environmental cost.

11. The method of claim 6 wherein the one or more computer networks are under control of a first entity, wherein the first entity is one of multiple clients of a configurable network service under control of a distinct second entity, the configurable network service enabling the clients to each configure one or more of multiple virtual computer networks provided by the configurable network service on behalf of the clients, and wherein the virtual computer network is one of the multiple virtual computer networks provided by the configurable network service.

12. The method of claim 11 wherein the one or more computer networks are at a first geographical location, and wherein the virtual computer network is at a remote second geographical location and is a virtual extension to at least one of the one or computer networks.

13. The method of claim 11 wherein the one or more computer networks are at a first geographical location, and wherein the virtual computer network includes computing nodes at multiple geographical locations distinct from the first geographical location.

14. The method of claim 11 wherein, after the adding of the one or more additional computing nodes to the group that are part of the remote virtual computer network, the group of multiple computing nodes includes one or more computing nodes under control of the first entity and includes computing nodes under control of multiple distinct remote network-accessible services that are provided for use by the first entity.

15. The method of claim 11 wherein the one or more computer networks are at a first geographical location, wherein the virtual computer network is at a remote second geographical location, and wherein the one or more computing devices providing the load balancing capabilities are at the remote second location and are provided to the first entity by the configurable network service for use in providing the load balancing capabilities for the group of multiple computing nodes.

16. The method of claim 11 wherein the one or more additional computing nodes are each a virtual machine hosted on one of multiple physical computing systems of the configurable network service, and wherein the adding of the one or more additional computing nodes to the group includes configuring one or more virtual machine manager modules that execute on one or more of the physical computing systems to manage communications for the hosted virtual machines so that the hosted virtual machines are included as part of the virtual computer network.

17. The method of claim 6 wherein the selecting for each of the received additional requests of a computing node of the modified group to respond to the additional request includes forwarding the additional request to the selected computing node to initiate providing of the type of functionality from the selected computing node to a computer system from which the additional request is received, and wherein the method further comprises, under control of one of the at least one additional computing nodes that is selected for the at least one additional requests, receiving the at least one additional requests that are forwarded from the one or more computing devices, and in response to each of the at least additional requests, providing the type of network-accessible functionality to a computing system that is not part of the group that initiated the additional request.

18. The method of claim 6 wherein the initiating of the modification to the group further includes initiating providing one or more specified types of computing-related resources as part of the remote virtual computer network for use by the added one or more additional computing nodes.

19. The method of claim 6 wherein the multiple computing nodes of the group are associated with a first user, wherein the first user specifies the defined criteria for the group before the receiving of the one or more requests, and wherein the one or more defined criteria for the group are further based in part on demand that is predicted by the first user for use of computing resources by the computing nodes of the group.

20. The method of claim 6 wherein the one or more defined criteria for the group are based on the status information for the one or more computing nodes indicating that a computing load of the one or more computing nodes has exceeded one or more defined thresholds.

21. The method of claim 6 wherein the one or more defined criteria for the group are based on the status information for the one or more computing nodes indicating a current level of use of one or more types of computing resources by the one or more computing nodes, the current level of use of the one or more types of computing resources by the one or more computing nodes including one or more of a volume of requests being received by each of the one or more computing nodes, a degree of network utilization of each of the one or more computing nodes, a degree of network latency for each of the one or more computing nodes, a degree of latency in responding to requests for each of the one or more computing nodes a degree of processor utilization for each of the one or more computing nodes, a degree of memory utilization for each of the one or more computing nodes, and a degree of input/output bandwidth utilization for each of the one or more computing nodes.

22. The method of claim 6 further comprising automatically obtaining the status information for the one or more computing nodes before the determining that the one or more defined criteria for the group are satisfied.

23. The method of claim 6 wherein the group of computing nodes has multiple associated criteria and has multiple associated actions, wherein each of the actions are associated with at least one of the multiple associated criteria such that the action is to be performed when the at least one associated criteria for that action are satisfied, and wherein one of the multiple associated actions is to perform the adding of the one or more additional computing nodes to the group from the remote virtual computer network, the one action being associated with the one or more defined criteria.

24. The method of claim 6 further comprising, after the initiating of the modification to the group of computing nodes, automatically determining that one or more other defined criteria for the group are satisfied, and initiating a further modification to the group of computing nodes that includes removing from the group one or more computing nodes that are part of the virtual computer network.

25. A method for a computing system to provide load balancing capabilities, the method comprising:

receiving, by a load balancer device, configuration information for a group of multiple computing nodes that are part of a physical network of a first user, the received configuration information indicating a remote configurable virtual computer network that is an extension to the physical network, the multiple computing nodes of the group being alternatives for providing a common type of network-accessible functionality to other computing systems and each executing a copy of a software application for use in the providing of the common type of functionality, the remote virtual computer network being provided to the first user by a remote configurable network service operated by an entity distinct from the first user, the configuration information further indicating defined criteria for the group of multiple computing nodes for use in determining when to use the indicated virtual computer network as part of modifying a quantity of the multiple computing nodes of the group, the defined criteria being based at least in part on current status information for one or more of the computing nodes of the group that indicates that a load of the one or more computing nodes has exceeded a defined threshold; and

under control of the load balancer device after it is configured with the received configuration information, providing load balancing capabilities for the group of multiple computing nodes by;

for each of one or more requests received from one or more of the other computing systems for the provided common type of functionality, selecting one of the computing nodes of the group to respond to the request and forwarding the request to the selected one computing node;

after the receiving of the one or more requests, automatically determining that the defined criteria for the group are satisfied, automatically determining one or more modifications to the group of computing nodes that will add one or more additional computing nodes to the group, and automatically initiating the determined modifications to the group of computing nodes by interacting with a programmatic interface of the configurable network service to cause the one or more additional computing nodes to be added to the indicated virtual computer network in such a manner that the additional computing nodes are available for use with the group; and

for each of one or more additional requests that are received after the automatic initiating of the determined modifications to the group, the one or more additional requests being received from one or more of the other computing systems for the provided common type of functionality, selecting one of the computing nodes of the modified group to respond to the additional request and forwarding the request to the selected one computing node, such that at least one of the additional computing nodes is selected for at least one of the additional requests.

28. A computing device configured to provide load balancing capabilities, comprising:

one or more computer processors; and

a load balancer module that is configured to, when executed by at least one of the one or more computer processors, automatically provide load balancing capabilities for functionality available from a group of computing nodes of a computer network, by

receiving at least one request for the available functionality, and selecting at least one of the computing nodes of the group to handle the at least one request;

automatically determining to modify a quantity of the computing nodes of the group based at least in part on one or more defined criteria for the group being satisfied, and initiating the determined modifying, the determined modifying including modifying use as part of the group of one or more computing nodes of a virtual computer network associated with the computer network; and

after the initiating of the determined modifying of the quantity of the computing nodes, receiving at least one additional request for the available functionality, and selecting at least one of the computing nodes of the group to handle the at least one additional request, such that at least one of the one or more computing nodes of the virtual computer network whose use as part of the group is modified is selected for the at least one additional request.

29. The computing device of claim 28 wherein the computing nodes of the group are associated with a first entity that specifies the defined criteria for the group, wherein the defined criteria are based at least in part on status information obtained about one or more of the computing nodes that are part of the group before the initiating of the determined modifying, wherein the determined modifying of the quantity of the computing nodes includes adding the one or more computing nodes of the virtual computer network to the group, wherein the computing nodes of the group each execute a copy of one or more software applications to provide the available functionality, and wherein the received at least one request and the received at least one additional request are from one or more computing systems that are not part of the group to obtain the available functionality from the one or more software applications.

30. The computing device of claim 28 wherein the computing nodes that are part of the group before the initiating of the determined modifying are at a first geographical location and are under control of a first entity, wherein the virtual computer network is a virtual extension to the computer network at a remote second geographical location and is under control of a distinct second entity, wherein the second entity provides a network-accessible service that enables clients to configure virtual computer networks provided by the network-accessible service on behalf of the clients, wherein the first entity is one of the clients of the network-accessible service, and wherein the virtual computer network extension is one of the virtual computer networks provided by the network-accessible service.

31. The computing device of claim 28 wherein the modifying of use as part of the group of the one or more computing nodes of the virtual computer network includes adding the one or more computing nodes of the virtual computer network to the group, and wherein the group of computing nodes did not include any computing nodes of the virtual computer network before the adding of the one or more computing nodes.

32. The computing device of claim 28 wherein the modifying of use as part of the group of the one or more computing nodes of the virtual computer network includes removing the one or more computing nodes of the virtual computer network from the group, and wherein the group of computing nodes does not include any computing nodes of the virtual computer network after the removing of the one or more computing nodes.

33. The computing device of claim 28 wherein the load balancer module includes software instructions for execution by the computing device.