Managing communications for modified computer networks

• US 8,194,680 B1
• Filed: 03/11/2009
• Issued: 06/05/2012
• Est. Priority Date: 03/11/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Brandwine, Eric J., Cohn, Daniel T., Doane, Andrew J.
• Primary Examiner(s): Ngo, Ricky
• Assistant Examiner(s): KAO, WEI PO ERIC
• Application Number:
  US12/402,331
• Time in Patent Office: 1,182 Days
• Field of Search: 370/352, 370/353, 370/354, 370/389, 370/392, 370/393, 370/396, 370/397, 370/398, 370/399, 370/401, 370/474, 370/475, 370/492, 370/501
• US Class Current: 370/398
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/50 Network service management, e.g. ensuring proper service fulfilment according to agreements
• CPC Class Codes: H04L 61/103 across network layers, e.g. resolution of network layer into physical layer addresses or address resolution protocol [ARP]
• CPC Class Codes: H04L 61/251 between different IP versions
• CPC Class Codes: H04L 69/167 Adaptation for transition between two IP versions, e.g. between IPv4 and IPv6 translation of Internet protocol [IP] addresses H04L61/2503
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 80/045 involving different protocol versions, e.g. MIPv4 and MIPv6

Abstract

Techniques are described for managing communications between multiple computing nodes, such as computing nodes that are part of a virtual computer network. In some situations, various types of modifications may be made to one or more computing nodes of an existing virtual computer network, and the described techniques include managing ongoing communications for those computing nodes so as to accommodate the modifications. Such modifications may include, for example, migrating or otherwise moving a particular computing node that is part of a virtual network to a new physical network location, or modifying other aspects of how the computing node participates in the virtual network (e.g., changing one or more virtual network addresses used by the computing node). In some situations, the computing nodes may include virtual machine nodes hosted on one or more physical computing machines or systems, such as by or on behalf of one or more users.

Claims

1. A method for a configured computing system to manage communications for modified computer networks, the method comprising:

for each of multiple computing nodes within a first virtual computer network overlaid on a physical substrate network, associating a virtual network address for the first virtual computer network with the computing node and generating mapping information that maps the computing node to a distinct substrate network address corresponding to a network location of the computing node in the physical substrate network;

modifying, by a communication manager module executing on the configured computing system that is configured to use at least the generated mapping information, an outgoing communication sent to a first computing node of the multiple computing nodes from another of the multiple computing nodes, the modifying including changing a specified destination address for the outgoing communication from a first virtual network address associated with the first computing node to be a first substrate network address corresponding to a first substrate network location of the first computing node;

forwarding the modified communication to the first computing node over the physical substrate network based at least in part on the first substrate network address included in the modified communication;

determining, by the configured computing system, to move the first computing node to a different second network location in the physical substrate network, the determining being based at least in part on identifying one or more benefits associated with moving the first computing node;

receiving, by the executing communication manager module, and after the forwarding of the modified outgoing communication, information indicating that the first computing node is now associated with a second substrate network address corresponding to the different second network location in the physical substrate network, the moved first computing node retaining the associated first virtual network address after the move;

updating, by the executing communication manager module, the generated mapping information to map the first computing node to the second substrate network address;

modifying, by the executing communication manager module configured to use at least the updated generated mapping information, an additional outgoing communication sent to the first computing node to change a specified destination address for the additional communication from the first virtual network address to the changed second substrate network address; and

forwarding the modified additional communication over the physical substrate network to the moved first computing node at the second substrate network location based at least in part on the changed second substrate network address included in the modified additional communication.

2. The method of claim 1 wherein the communication manager module manages a first physical sub-network of the physical substrate network to which the one or more of the multiple computing nodes belong, and wherein the method further comprises:

managing, by the executing communication manager module, a move of a second of the multiple computing nodes from a network location of the physical substrate network that is not part of the first physical sub-network to a changed location that is part of the first physical sub-network, the moving of the second computing node including updating the generated mapping information to map the moved second computing node to a changed substrate network address that corresponds to the changed location in the first physical sub-network, the associated virtual network address for the moved second computing node being unchanged after the move; and

managing another modified communication that is received for the moved second computing node over the physical substrate network after the another modified communication is routed over the physical substrate network using the changed substrate network address for the second computing node, the managing of the another modified communication including further modifying the another communication to replace the changed substrate network address for the second computing node with the unchanged virtual network address associated with the second computing node, and providing the further modified another communication to the second computing node.

3. The method of claim 1 wherein the executing communication manager module is one of multiple communication manager modules of an overlay network manager system that each manages an associated subset of the multiple computing nodes, and wherein a system manager module of the overlay network manager system maintains the generated mapping information for the first virtual computer network and maintains mapping information for each of multiple other virtual computer networks that are provided by the overlay network manager system on behalf of multiple remote users.

4. A computer-implemented method for managing communications for modified computer networks, the method comprising:

modifying, by a computing system configured to use mapping information for a first virtual computer network, an outgoing communication sent to a first computing node of multiple computing nodes in the first virtual computer network from another of the multiple computing nodes, the modifying of the outgoing communication including changing a destination of the outgoing communication from a first virtual network address associated with the first computing node to a distinct first substrate network address that corresponds to a location of the first computing node in a substrate network on which the first virtual computer network is overlaid;

forwarding the modified communication to the first computing node over the substrate network based at least in part on the first substrate network address;

determining, by the configured red computing system, to move the first computing node to a different location in the substrate network, the different location corresponding to a second substrate network address that is distinct from the first substrate network address, the determining being based at least in part on identifying one or more benefits associated with moving the first computing node;

managing, by the configured computing system, and after the forwarding of the modified communication, the move of the first computing node to the different location in the substrate network, the moved first computing node retaining the first virtual network address, the managing of the move including updating the mapping information to associate the first computing node with the second substrate network address;

modifying, by the configured computing system, and after the move of the first computing node, an additional outgoing communication sent to the first computing node from another of the multiple computing nodes, the modifying of the additional outgoing communication including changing a destination of the additional outgoing communication from the first virtual network address to the second substrate network address based at least in part on the updated mapping information; and

forwarding the modified additional outgoing communication to the moved first computing node over the substrate network based at least in part on the second substrate network address.

5. The method of claim 4 wherein the first computing node is provided by a first physical computing system at a first network location of the substrate network before the move of the first computing node, wherein the move of the first computing node to the different location includes performing a logical move to a distinct second physical computing system at the different location that represents the first computing node after the move, wherein the determining to move the first computing node includes determining to move the first computing node to the second physical computing system, and wherein the identified one or more benefits associated with moving the first computing node include one or more benefits from using the second physical computing system to provide the first computing node.

6. The method of claim 5 wherein the identified one or more benefits include at least one benefit selected from a group of benefits including operation of the second physical computing system having a lower cost than operation of the first physical computing system, of the second physical computing system having one or more capabilities of interest, of use of the second physical computing system providing improved utilization of computing resources relative to use of the first physical computing system, of use of the second physical computing system providing improved utilization of network bandwidth relative to use of the first physical computing system, of use of the second physical computing system including consolidation of multiple computing nodes at the second physical computing system and enabling suspension of use of the first physical computing system to conserve power, of operating problems that are identified for the first physical computing system and that are not identified for the second physical computing system, and of a different use being identified for the first physical computing system that has higher priority than providing the first computing node.

7. The method of claim 5 wherein the performing of the logical move of the first computing node from the first physical computing system to the second physical computing system includes at least one of executing one or more software programs on the second physical computing system that were previously executing on the first physical computing system and of copying data being used by the first computing node from the first physical computing system to the second physical computing system.

8. The method of claim 4 wherein the computing system is further configured to operate as part of an overlay network manager system that provides multiple virtual computer networks to multiple clients, wherein the first virtual computer network is provided by the overlay network manager system on behalf of one of the multiple clients, and wherein the determining to move the first computing node is performed based at least in part on a request from the one client to modify operations of the first virtual computer network in a specified manner.

9. The method of claim 4, further comprising:

managing one or more modifications to the first virtual computer network that include an association of a new virtual network address with a second of the multiple computing nodes that is distinct from a previous virtual network address used by the second computing node before the one or more modifications, the second computing node having an associated substrate network address before the one or more modifications that corresponds to a location of the second computing node in the substrate network and that is not changed for the second computing node by the one or more modifications; and

managing another outgoing communication that is sent by the second computing node to at least one other of the multiple computing nodes and that specifies the new virtual network address of the second computing node to indicate a source of the another outgoing communication, the managing of the another outgoing communication including modifying the another outgoing communication to change the new virtual network address in the modified another outgoing communication to the associated substrate network address for the second computing node.

10. The method of claim 9 wherein the association of the new virtual network address with the second computing node corresponds to at least one of replacing the previous virtual network address with the new virtual network address and of using the new virtual network address in addition to the previous virtual network address as part of at least two distinct network interfaces for the second computing node, and wherein the managing of the one or more modifications includes updating the mapping information for the first virtual computer network to include an association of the second computing node with the new virtual network address.

11. The method of claim 4 wherein the modifying of the outgoing communication and the modifying of the additional outgoing communication are performed by one or more communication manager modules that are associated with one or more of the multiple computing nodes that initiate sending of the outgoing communication and that initiate sending of the additional outgoing communication, the one or more communication manager modules being remote from the first computing node.

12. The method of claim 11 wherein the managing of the move of the first computing node to the different location is performed by a system manager module that maintains the mapping information for the first virtual computer network, and wherein the updating of the mapping information includes providing updated mapping information from the system manager module to the one or more communication manager modules for use in updating a locally stored copy of at least some of the mapping information.

13. The method of claim 4 wherein the substrate network is a physical network that includes multiple interconnected networking devices, and wherein the multiple interconnected networking devices use substrate network address information from the mapping information and the updated mapping information that is included in multiple communications to effectuate forwarding of the multiple communications over the substrate network, the forwarding of the multiple communications including the forwarding of the modified outgoing communication to the first computing node over the physical substrate network and the forwarding of the modified additional outgoing communication to the moved first computing node over the physical substrate network.

14. The method of claim 13 wherein, after the move of the first computing node, the substrate network is expanded to interconnect additional computing nodes at additional locations, and wherein the method further comprises managing one or more additional modifications to the first virtual computer network that include moving one or more of the multiple computing nodes to use one or more of the additional computing nodes at the additional locations.

15. The method of claim 4 wherein the move of the first computing node to the different location in the substrate network includes moving the computing node from a first data center at a first geographical location to a second data center at a second geographical location.

16. The method of claim 15 wherein the first and second data centers are connected using one or more public networks.

17. The method of claim 15 wherein the first and second data centers are connected using a private connection.

18. The method of claim 4 wherein the substrate network connects computing nodes that are part of multiple distinct networking layer 2 sub-networks, and wherein the move of the first computing node to the different location in the substrate network includes moving the first computing node from a first networking layer 2 sub-network at a first location to a second networking layer 2 sub-network at a second location.

19. The method of claim 4 wherein the move of the first computing node to the different location in the substrate network includes selecting the different location, the selected different location being remote from an initial location of the first computing node such that additional network latency occurs for the forwarding of the modified additional communication to the moved first computing node at the different location relative to the forwarding of the modified communication to the first computing node at the initial location, the selecting being performed without regard to the additional network latency that occurs for the different location.

20. The method of claim 4 wherein the multiple computing nodes are each a virtual machine hosted on one of multiple host physical computing systems, wherein the configured computing system is a first of the host physical computing systems and executes a virtual machine communication manager module to manage communications by the virtual machine computing nodes hosted on the first host physical computing system, and wherein the virtual machine communication manager module is one of multiple communication manager modules of an overlay network manager system that provides multiple virtual computer networks to multiple users that are each overlaid on the substrate network.

21. A non-transitory computer-readable medium whose contents configure a computing system to manage communications for modified computer networks, by performing a method comprising:

managing one or more sent communications that involve a first of multiple computing nodes of a first virtual computer network and that each specify a virtual network address associated with the first computing node for the first virtual computer network, the managing of each of the one or more sent communications including identifying a distinct substrate network address for the first computing node that is for use in delivering the sent communication to a network location of the first computing node in a physical substrate network interconnecting the multiple computing nodes, the substrate network address being identified based at least in part on a mapping for the first computing node of the associated virtual network address to the substrate network address; and

determining to initiate one or more modifications to the first virtual computer network in such a manner as to enable delivery of additional sent communications that involve the first computing node, the one or more modifications including a change to the first computing node that causes a modification of at least one of the associated virtual network address for the first computing node and of the substrate network address for the first computing node and further including updating the mapping for the first computing node to reflect the change to the first computing node; and

after the managing of the one or more sent communications, managing the delivery of the additional sent communications, the managing of the delivery including modifying the additional sent communications based on the updated mapping.

22. The non-transitory computer-readable medium of claim 21 wherein the change to the first computing node causes a modification of the substrate network address for the first computing node, wherein the additional sent communications are outgoing communications sent to the first computing node by one or more other computing nodes, and wherein the modifying of the additional sent communications includes using the updated mapping to identify the modified substrate network address of the first computing node for use as a destination network address for the additional sent communications.

23. The non-transitory computer-readable medium of claim 22 wherein the change to the first computing node involves a move of the first computing node to a new network location in the physical substrate network such that the modified substrate network address corresponds to the new network location, wherein the move of the first computing node is performed in a manner transparent to other computing nodes of the first virtual computer network, wherein the additional sent communications each specify the virtual network address associated with the first computing node to indicate that the first computing node is an intended destination of the additional communication, and wherein the modifying of the additional communications includes changing the specified virtual network address in each additional communication to the modified substrate network address, and wherein the managing of the delivery of the additional sent communications includes forwarding the modified additional communication to the moved first computing node over the physical substrate network by using the modified substrate network address to route the modified additional communication.

24. The non-transitory computer-readable medium of claim 21 wherein the change to the first computing node causes a modification of the associated virtual network address of the first computing node for the first virtual computer network, wherein the additional sent communications are outgoing communications sent to one or more other computing nodes by the first computing node and each include an indication of the modified virtual network address to identify the first computing node, and wherein the delivery of the additional sent communications includes using the updated mapping to identify the substrate network address of the first computing node as being associated with the modified virtual network address and using the identified substrate network address as a source network address for the additional sent communications.

25. The non-transitory computer-readable medium of claim 21, wherein the change to the first computing node causes a modification of the associated virtual network address of the first computing node for the first virtual computer network, wherein the additional sent communications are outgoing communications that each specify the modified virtual network address associated with the first computing node to indicate that the first computing node is an intended destination, wherein the managing of the delivery of the additional sent communications including using the updated mapping to identify the substrate network address for the first computing node as corresponding to the specified modified virtual network address, wherein the modifying of the additional communication includes changing the specified modified virtual network address in the additional communication to the identified substrate network address, and wherein the managing of the delivery of the additional sent communications further includes forwarding the modified additional communication to the first computing node over the physical substrate network by using the identified substrate network address to route the modified additional communication.

26. The non-transitory computer-readable medium of claim 21 wherein the computer-readable medium is a memory of the configured computing system on which the contents are stored, and wherein the contents are instructions that when executed cause the configured computing system to perform the method.

27. A system, comprising:

one or more processors of one or more computing systems; and

one of multiple communication manager modules that manage communications between multiple computing nodes of a first virtual computer network via one or more intermediate physical networks, the one communication manager module being associated with a first group of one or more of the multiple computing nodes that are implemented as virtual machines hosted by one or more computing systems, each of the communication manager modules other than the one communication manager module having a distinct associated group of one or more other of the multiple computing nodes, the one communication manager module being configured to, when executed by at least one of the one or more processors, manage outgoing communications for the computing nodes of the first group by;

after one of the computing nodes of the first group initiates sending a communication to a destination first computing node that is part of the associated group for a first other of the communication manager modules, determining a destination network address in the one or more intermediate physical networks for use with the communication, the determined destination network address corresponding at least in part to the first other communication manager module, and using the determined destination network address to forward the sent communication to the first computing node via the one or more intermediate physical networks;

determining to initiate a move of the first computing node to a new location in the one or more intermediate physical networks such that the first computing node becomes part of the associated group for a distinct second other of the communication manager modules;

after the first computing node is moved to the new location, determining a distinct second destination network address in the one or more intermediate physical networks, the determined destination network address corresponding at least in part to the second other communication manager module, and updating mapping information associated with the first virtual computer network to include an indication of the determined second destination network address for the first computing node; and

after one of the computing nodes of the first group initiates sending a second communication to the first computing node, using the determined second destination network address to forward the second communication to the moved first computing node via the one or more intermediate networks by modifying the second communication to reflect the determined second destination network address, the determined second destination network address being identified based on the updated mapping information.

28. The system of claim 27 wherein the one or more computing nodes of the first group include multiple computing nodes that are each one of multiple virtual machines hosted by a first computing system, wherein the first computing system further executes the one communication manager module, wherein the system further comprises the first computing system, and wherein the first computing node is implemented as a virtual machine hosted by a second computing system while the first computing node is part of the associated group for the first other communication manager module, and is implemented as a virtual machine hosted by a distinct third computing system while the first computing node is part of the associated group for the second other communication manager module.

29. The system of claim 27 wherein the one or more physical intermediate networks are a physical substrate network that interconnects the multiple computing nodes and on which the first virtual computer network is overlaid, wherein the multiple communication manager modules are part of an overlay network manager system that provides the first virtual computer network and that automatically manages the move of the first destination computing node to the new location, and wherein the move of the first destination computing node to the new location is performed in a manner that is transparent to the multiple computing nodes other than the first computing node.

30. The system of claim 29 wherein the determining by the one communication manager module of the destination network address for use with the sent communication includes interacting with a remote system manager module of the overlay network manager system that maintains the mapping information, wherein the mapping information includes substrate network addresses corresponding to network locations of the multiple computing nodes in the physical substrate network, wherein the determined destination network address for use with the sent communication is the substrate network address for the first destination computing node, the substrate network address for the first destination computing node being for use in routing the sent communication over the physical substrate network to the first computing node, and wherein the determined second destination network address for use with the sent second communication is a changed substrate network address for the first computing node that corresponds to the new location in the physical substrate network of the moved first computing node, the changed substrate network address for the moved first computing node being for use in routing the sent second communication over the physical substrate network to the moved first computing node.

31. The system of claim 27 wherein the one communication manager module includes software instructions for execution by the at least one processors of the one or more computing systems.

32. The system of claim 27 wherein the one communication manager module consists of a means for managing outgoing communications for the computing nodes of the first group by:

after one of the computing nodes of the first group initiates sending a communication to a destination first computing node that is part of the associated group for a first other of the communication manager modules, automatically determining a destination network address for use with the communication that corresponds at least in part to the first other communication manager module, and using the determined destination network address to forward the sent communication to the first computing node via the one or more intermediate networks; and

after the first computing node is moved to a new location and becomes part of the associated group for a distinct second other of the communication manager modules, and after one of the computing nodes of the first group initiates sending a second communication to the first computing node, automatically determining a distinct second destination network address for use with the second communication that corresponds at least in part to the second other communication manager module, and using the determined second destination network address to forward the second communication to the moved first computing node via the one or more intermediate networks.