CROSS-REGIONAL VIRTUAL NETWORK PEERING
1. A virtual network management device operable to implement peering of virtual networks, the device comprising:
- a memory containing machine-readable instructions; and
a processor to execute the machine-readable instructions to;
determine unique regional identifiers for virtual networks located in two different geographical regions; and
generate an encapsulated packet for transmission from a first virtual machine (VM) in a first one of the virtual networks located in a first region of the geographical regions to a second VM in a second one of the virtual networks located in a second region of the geographical regions,wherein the encapsulated packet includes a header comprising the regional identifiers, virtual network addresses of the first and second VMs, and physical network addresses of the first and second VMs.
Virtual networks located in different regions of cloud provider are peered using unique regional identifiers for the virtual networks. The regional identifiers and other information are pushed down a network management stack to implement the peering.
- 1. A virtual network management device operable to implement peering of virtual networks, the device comprising:
a memory containing machine-readable instructions; and a processor to execute the machine-readable instructions to; determine unique regional identifiers for virtual networks located in two different geographical regions; and generate an encapsulated packet for transmission from a first virtual machine (VM) in a first one of the virtual networks located in a first region of the geographical regions to a second VM in a second one of the virtual networks located in a second region of the geographical regions, wherein the encapsulated packet includes a header comprising the regional identifiers, virtual network addresses of the first and second VMs, and physical network addresses of the first and second VMs.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 20)
- 10. A computer-implemented method for peering virtual networks managed by a cloud provider, the method comprising:
determining unique regional identifiers for first and second virtual networks located in two different geographical regions based on regional information pushed down a network management stack of the cloud provider from a regional layer to a host layer; and generating an encapsulated packet for transmission from a first virtual machine (VM) in the first virtual network located in a first region of the geographical regions to a second VM in the second virtual network located in a second region of the geographical regions, wherein the encapsulated packet includes a header comprising the regional identifiers, virtual network addresses of the first and second VMs, and provider network addresses of the first and second VMs.
- View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
- 19. One or more non-transitory computer-readable media devices having computer-executable instructions embodied thereon, wherein the instructions are executed by at least one processor to peer virtual networks of a tenant of a cloud provider provided in different geographical regions of the cloud provider'"'"'s network, and the computer-executable instructions are executable by at least one processor to:
determine regional information of the virtual networks; determine unique regional identifiers for the virtual networks based on the regional information; subscribe to mappings of addresses in a customer address space of the tenant to a provider address space of the cloud provider; and generate an encapsulated packet for transmission from a first virtual machine (VM) in a first one of the virtual networks located in a first region of the geographical regions to a second VM in a second one of the virtual networks located in a second region of the geographical regions based on the unique regional identifiers and the subscribed mappings.
Large-scale networked systems are commonplace platforms employed in a variety of settings for running applications and maintaining data for business and operational functions. For instance, a data center (e.g., physical cloud computing infrastructure) may provide a variety of services (e.g., web applications, email services, search engine services, etc.) for a plurality of customers simultaneously. These large-scale networked systems typically include a large number of resources distributed throughout one or more data centers which may be located in one or more geographical regions, and may be implemented by a cloud provider providing cloud services to customers.
The implementation of connections between endpoints of remote networks in these large-scale networked systems is often rooted in physical connectivity, but there is an emerging trend toward virtualizing connectivity. One instance of virtualizing connectivity includes a virtualized network which may involve creating a logical isolation of cloud resources dedicated to a customer'"'"'s cloud service. The virtualized network may enable cloud resources, including physical machines or virtual machines, that are assigned to a customer to securely communicate with each other and be isolated from other customer virtualized networks.
A virtualized network may connect to another virtualized network or to an on-premises network of a customer through the Internet or private networks. A gateway may be used to securely provide the connections between the virtualized networks or to the on-premises network. One type of gateway may use virtual private network (VPN) technology, such as Internet Protocol Security (IPsec)/Internet Key Exchange (IKE) VPN tunnels, to send encrypted traffic across a public connection to an on-premises location or to send encrypted traffic between virtualized networks over the cloud provider'"'"'s network.
The gateway may be beneficial to provide secure connections. However, the gateway may become a bottleneck point. For example, a gateway may have bandwidth constraints that can limit the amount of traffic it can send or receive at any given time. As a result, customers that utilize cloud services may not able to push a significant amount of data needed for their organizations through the gateway, and may not be able to execute real-time applications that require exchanging a large amount of data due to latency caused by bandwidth constraints of the gateway.
Embodiments and examples are described in detail in the following description with reference to the following figures. The embodiments are illustrated by examples shown in the accompanying figures in which like reference numerals indicate similar elements.
For simplicity and illustrative purposes, the principles of the present disclosure are described by referring mainly to embodiments and examples thereof. In the following description, numerous specific details are set forth in order to provide an understanding of the embodiments and examples. It will be apparent, however, to one of ordinary skill in the art, that the embodiments and examples may be practiced without limitation to these specific details. In some instances, well known methods and/or structures have not been described in detail so as not to unnecessarily obscure the description of the embodiments and examples. Furthermore, the embodiments and examples may be used together in various combinations.
According to an embodiment of the present disclosure, global virtual network peering facilitates communication between virtual networks which may be located in different geographical regions. The global virtual network peering may impose no additional bandwidth restrictions on communications between virtual networks and thus minimizes latency for the communications. Through use of the global virtual network peering, customers are able to communicate from a VM in one region to a VM in another region directly without the use of a gateway or an extra hop in-between. Furthermore, large customers of a cloud provider may have multi-region deployments in the cloud and often need to be able to communicate across geographical regions with high bandwidth and low latency, such as for data replication through private internet protocol (IP) addresses, and disaster recovery. Global virtual network peering provides secured global connectivity for VMs across the regions with high performance. In addition, private, cross-regional connectivity through Internet Protocol version 6 (IPv6) or another IP version may be employed on the cloud provider backbone to provide enhanced IP-based security for cross-regional connectivity. For example, IPv6 prefixes may be used to distinguish IP regions in different regions. Use of IPv6 for global virtual network peering enables more efficient routing, more efficient routing packet processing, directed data flows, simplified network configurations, and IP Security (Sec). However, IP versions, other than IPv6, may be used for global virtual network peering as long as regional identifiers may be used in the header.
The distributed computing environment of
Public network 101 may include data centers configured to host and support operations. It may be understood and appreciated that data center 114 and data center 139 shown in
Data center 114 illustrates a data center comprising a plurality of servers, such as servers 120-122. A fabric controller 130 is responsible for automatically managing the servers 120-122 and distributing tasks and other resources within the data center 114. By way of example, the fabric controller 130 may rely on a service model (e.g., designed by a customer that owns the modular-application) to provide guidance on how, where, and when to configure a server and how, where, and when to place an application, such as applications 116 and 118, on a server. In one embodiment, one or more role instances of a modular-application may be placed on one or more of the servers of data center 114, where the one or more role instances may represent the portions of software, component programs, or instances of roles describing operations to be performed by the modular-application.
Data center 139 illustrates a data center comprising a plurality of nodes, such as node 131 and node 134. One or more virtual machines may run on nodes of data center 139, such as virtual machine 136 of node 134 for example. Although
In operation, the virtual machines are dynamically assigned resources on a first node and second node of the data center, and endpoints (e.g., the role instances) are dynamically placed on the virtual machines to satisfy the current processing load. In one instance, a fabric controller 130 is responsible for automatically managing the virtual machines running on the nodes of the data center 139 and for placing the role instances and other resources (e.g., software components) within the data center 139. By way of example, the fabric controller 130 may rely on a service model (e.g., designed by a customer that owns the service application) to provide user interface on how, where, and when to configure the virtual machines, such as virtual machine 136, and how, where, and when to place the role instances thereon.
As discussed above, the virtual machines may be dynamically established and configured within one or more nodes of a data center. As illustrated herein, node 131 and node 134 may be any form of computing devices, such as, for example, a personal computer, a desktop computer, a laptop computer, a mobile device, a consumer electronic device, server(s), and the like. In one instance, the nodes host and support the operations of the virtual machines, while simultaneously hosting other virtual machines carved out for supporting other tenants of the data center 139, such as internal services 138 and hosted services 140. Often, the role instances may include endpoints of distinct service applications owned by different customers. The data centers may include storage 141 for storing any information.
Typically, each of the nodes includes, or is linked to, some form of a computing unit (e.g., a central processing unit, microprocessor, etc.) to support operations of the component(s) running thereon. As utilized herein, the phrase “computing unit” generally refers to a dedicated computing device with processing power and storage memory, which supports operating software that underlies the execution of software, applications, and computer programs thereon. In one instance, the computing unit is configured with tangible hardware elements, or machines, that are integral, or operably coupled, to the nodes to enable each device to perform a variety of processes and operations. In another instance, the computing unit may encompass a processor (not shown) coupled to the computer readable medium (e.g., computer storage media and communication media) accommodated by each of the nodes.
The role instances that reside on the nodes support the operation of service applications and may be interconnected via APIs. In one instance, one or more of these interconnections may be established via a network cloud, such as public network 101. The network cloud serves to interconnect resources, such as the role instances, which may be distributable placed across various physical hosts, such as nodes 131 and 134. Also, the network cloud facilitates communication over channels connecting the role instances of the service applications running in the data center 139. By way of example, the network cloud may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the internet. Accordingly, the network is not further described herein.
A virtual network may include a logical isolation of the cloud resources dedicated to the customer'"'"'s subscription. The virtual network may be connected to other virtual networks, or to an on-premise network of the customer. A virtual network may be created by mapping a customer address space of resources in the virtual network to a physical address space of the physical resources supporting the virtual network. For example, each VM in the virtual network is given two IP addresses: one IP address—the Customer Address (CA)—is visible in the virtual network and is relevant in the context of a given virtual subnet in the virtual network; and the other IP address—the Provider Address (PA)—is relevant in the context of the physical network in the cloud datacenter. The PA may also be referred to as a physical address in a physical address space of the provider network, such as an IPv6 address of a host. A subnet of a virtual network may include a subset of the customer addresses of the customer address space of the virtual network. The decoupling of the tenant and datacenter IP addresses provides many benefits. One benefit is that a tenant can move their network and subnets to VMs in the cloud without modifying the physical network configuration and without worrying about what other tenants or virtual networks are present in that datacenter. Another benefit is policy enforcement in the VM hosts, which provides a scalable solution for multi-tenant isolation. Also, a VM may be migrated to a new physical host without requiring notification to the customer. In other words, because each VM in the virtual network has two IP addresses, the IP address that is relevant in the datacenter context—i.e., the Physical Address—can be changed without needing to change the IP address that is in the VM—i.e., the Customer Address. Therefore, when a client is talking to the VM via the CA, the client is unaware that the VM has physically moved to a different subnet. Additionally, using distributed routing domains, the VM is unaware that its IP address has been virtualized. From the VM'"'"'s perspective, all communication is occurring via the CA IP address.
The virtual networks A-B may be connected through global virtual network peering as is further described below. It should be understood that connecting the virtual networks A-B through global virtual network peering may include connecting subnets in the virtual networks A-B through global virtual network peering.
The network management stack 300 includes network regional layer (NRL) 301 for managing network resources at the regional level, including network resources for virtual networks deployed in a region. The NRL 301 may include a network regional agent (NRA) 311 comprising deployment and configuration tools that allow a customer to manage their network resources for an entire region. For example, the cloud provider may provide network resources, such as NICs, public IP addresses, and VMs, that are located in a data center in a particular geographic region. These network resources may be managed independently or collectively by the customer through the NRA 311 using a region ID and other regional information for the network resources. The cloud provider may have network resources in multiple regions and each region may have a region name, such as “West US”, “East US”, etc. A virtual network for a customer may be created in a particular region, and the NRA 311 may associate and store a VNET ID and region ID (e.g., region name) for a virtual network created in a particular region. Also, the NRA 311 may store a source IPv6 prefix for the region. For example, IPv6 prefixes are used to uniquely identify the regions of the cloud provider. The NRA 311 may store, along with the VNET ID and the region ID, the IPv6 prefix that uniquely identifies the region. The NRA 311 may also receive parameters and instructions for the network resources via a customer portal 350 or API 351. The parameters may specific various properties for the network resources, such as tags, network security groups, etc., and the instructions may include instructions for managing the network resources, such as assigning the network resources to an application, creating virtual networks, specifying virtual subnets to connect through global virtual network peering, etc. The NRA 311 may also provide administrative information through the portal 350 or API 351, such as alerts for specific network monitoring metrics.
The NRA 311 manages network resources on a regional basis for virtual networks. For example, a virtual network may be created in a region. Virtual machines can use virtual networks that are in the same region. However, a virtual machine can'"'"'t be part of a virtual network that'"'"'s in a different region, which is enforced by the NRA 311. Other regional-based management of virtual networks may also be performed by the NRA 311.
The network management stack 300 includes network cluster layer (NCL) 302, which may include a network cluster agent (NCA) 312 that manages network resources and network state provisioning. The NCL 302 enables networking scenarios like virtual networks, load balancing, and Quality of Service (QoS) at the cluster level. A cluster may include VMs (nodes) that run on groups of physical servers known as “clusters” (e.g., of approximately 1,000 machines). Each cluster may be independently managed by a scaled-out and redundant platform fabric controller (FC) software component. For example, an FC manages the lifecycle of VMs and applications running in its cluster, including provisioning and monitoring the health of the hardware under its control. The FC executes both automatic operations, like healing VM instances to healthy servers when it determines that the original server has failed, as well as application-management operations like deploying, updating, reimaging and scaling out applications. Dividing the datacenter into clusters isolates faults at the FC level, preventing certain classes of errors from affecting servers beyond the cluster in which they occur.
The network management stack 300 includes a network host layer (NHL) 303. The NHL 303 may include network host agent (NHA) 313 at the host level. The NHA 313 may include a plugin loaded on a host. The NHA 313 performs management operations at the host level.
Assume that customer 160 wants to connect VNET A and VNET B through global virtual network peering. Customer 160 may be a subscriber to a cloud service provided by a cloud provider which implements data centers A-B in regions A-B, respectively. One example of a cloud service that customer 160 may subscribe to is infrastructure as a service (IaaS). For example, the cloud provider hosts the infrastructure components traditionally present in an on-premises data center, including servers, storage and networking hardware, as well as the virtualization or hypervisor layer. At 1, customer 160 sends a request to the cloud provider, such as through customer portal 350 or API 351 shown in
At 4, each of the NRAs 311A-B learns properties of the remote virtual network, such as its unique Generic Routing Encapsulation (GRE) key. The properties may be learned by exchanging VNET information with the remote NRA and based on information provided from the sub-agent. The NRAs 311A-B may also perform checks. For example, the customer addresses of VNET subnets being peered should not overlap. If there is overlap, then the customer may be notified and customer addresses (CAs) may be modified as needed.
At 5, the NRAs 311A-B transmit regional information associated with the VNETs A-B to the respective NCAs 312A-B that may be needed for peering the VNETs. This information may include region IDs of the VNETs, VNET IDs, storage that can be accessed by a particular VNET, other properties learned at step 4, etc. This information also includes the information for the remote VNET. After the NCAs 312A-B receive the regional information associated with the VNETs, the NCAs 312A-B may not need to communicate with each other to perform the peering.
Each of the regions A-B may maintain a pub/sub mechanism for determining mappings between the customer address space of the VNET (e.g., comprised of virtual network addresses) and the physical address space (e.g., comprised of physical network addresses). At 6, the NCAs 312A-B publish mappings for the customer addresses of the VNETs to their physical addresses (e.g., IPv6 physical addresses), and the mappings may then be published for the lower layers of the stack, such as NHAs 313A-B, to consume. For example, at steps 8-9, the NHAs 313A-B obtain the mappings for the local and remote VNETs A-B through their subscriptions to local and remote pub/sub mechanisms.
Also, at step 7, the NHAs 313A-B receive, from respective NCAs 312A-B, information, such as goal state for link records of VNETs A-B and remote region information, such as source IPv6 prefixes. Goal state may refer to goal state-based management of the customer'"'"'s virtual networks, such as goal states for security groups, security rules, public IP addresses, subnet configurations, etc. Furthermore, IPv6 prefixes may be used to uniquely identify a region.
After the NHAs 313A-B receive the information described above, such as with respect to steps 7-9, hosts A-B may communicate through global virtual network peering. For example, NHA 313A determines that VM1 (on host A) has CA1, PA1v6 and MAC1, and also determines that VM2 (on host B) has CA2, PA2v6 and MAC2. This information and the GRE key may be used to generate the encapsulated header 400, which is shown in
Global virtual network peering enables seamlessly connecting two virtual networks without being susceptible bandwidth constraints or other performance constraints caused by virtual network gateways. Once peered, the virtual networks may appear as one, for connectivity purposes. The traffic between virtual machines in the peered virtual networks may be routed through the cloud provider backbone infrastructure, much like traffic is routed between virtual machines in the same virtual network, through private IP addresses only. The peered virtual networks have non-overlapping IP address spaces. In an example, address ranges cannot be added to or deleted from the address space of a virtual network once a virtual network is peered with another virtual network. If a customer needs to add address ranges to the address space of a peered virtual network, peering is first removed, and then the addresses are added to the address space, and then the global virtual network peering is performed again. Also, virtual networks that exist in two different subscriptions may be peered, as long a privileged user (see specific permissions) of both subscriptions authorizes the peering, and the subscriptions are associated to the same active directory tenant.
The examples and embodiments described above may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and so forth, which perform particular tasks or implement particular abstract data types. The examples and embodiments described above may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in local and/or remote computer storage media including memory storage devices.
The computer 500 typically includes a variety of non-transitory computer-readable media 504. Computer-readable media 504 may be any available media that can be accessed by the computer 500 and includes both volatile and nonvolatile media, and removable and non-removable media, but excludes propagated signals. By way of example, and not limitation, computer-readable media 504 may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computer 500. Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above may also be included within the scope of computer-readable media. Computer-readable media may be embodied as a computer program product, such as software stored on computer storage media.
The data storage or system memory 502 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) and random access memory (RAM). A basic input/output system (BIOS), containing the basic routines that help to transfer information between elements within computer 500, such as during start-up, is typically stored in ROM. RAM typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 501. By way of example, and not limitation, data storage 502 holds an operating system, application programs, and other program modules and program data.
Data storage 502 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only, data storage 502 may be a hard disk drive that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive that reads from or writes to a removable, nonvolatile magnetic disk, and an optical disk drive that reads from or writes to a removable, nonvolatile optical disk such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The drives and their associated computer storage media, described above and illustrated in
A user may enter commands and information through a user interface 505 or other input devices such as a tablet, electronic digitizer, a microphone, keyboard, and/or pointing device, commonly referred to as mouse, trackball or touch pad. Other input devices may include a joystick, game pad, satellite dish, scanner, or the like. Additionally, voice inputs, gesture inputs using hands or fingers, or other natural user interface (NUI) may also be used with the appropriate input devices, such as a microphone, camera, tablet, touch pad, glove, or other sensor. These and other input devices are often connected to the processing unit 501 through a user input interface 505 that is coupled to the system bus 503, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 506 or other type of display device is also connected to the system bus 503 via an interface, such as a video interface. The monitor 506 may also be integrated with a touch-screen panel or the like. Note that the monitor and/or touch screen panel can be physically coupled to a housing in which the computing device 500 is incorporated, such as in a tablet-type personal computer. In addition, computers such as the computing device 500 may also include other peripheral output devices such as speakers and printer, which may be connected through an output peripheral interface or the like.
The computer 500 may operate in a networked or cloud-computing environment using logical connections to one or more remote devices, such as a remote computer. The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 500. The logical connections depicted in
When used in a networked or cloud-computing environment, the computer 500 may be connected to a public or private network through a network interface or adapter 507. In some embodiments, a modem or other means for establishing communications over the network. The modem, which may be internal or external, may be connected to the system bus 503 via the network interface 507 or other appropriate mechanism. A wireless networking component such as comprising an interface and antenna may be coupled through a suitable device such as an access point or peer computer to a network. In a networked environment, program modules depicted relative to the computer 500, or portions thereof, may be stored in the remote memory storage device. It may be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
The computer 500 may be a server or another type of computer. The computer 500 may include a platform for hosting one or more components of the network management stack 300 shown in
At 601, a cloud provider receives a request from a customer to peer virtual networks that are provided in different geographic regions. For example, the cloud provider receives a request to peer VNETs A-B via customer portal 350 or API 351 shown in
At 602, the cloud provider determines regional information for virtual networks specified in the request. The regional information may include a region ID and other regional information for the virtual networks. A virtual network for a customer may be created in a particular region, and an agent may associate and store a VNET ID and region ID (e.g., a region name) for a virtual network created in a particular region. Also, the agent may store a source IPv6 prefix for the region. For example, IPv6 prefixes are used to uniquely identify the regions of the cloud provider.
In an example, a network management agent of the cloud provider may determine the regions of the virtual networks. For example, a network management agent in the network regional layer 301 of the network management stack 300, e.g., client-facing sub-agents 321 A-B shown in
In the example shown in
At 603, agents at the regional layer 301 of the network management stack initiate linking of the VNETs specified in the request, for example, by obtaining shared information about the remote VNET from the remote agent managing the remote VNET. For example, the NRAs 311A-B share information about their VNETs, such as regions IDs, VNET IDs, etc. Also, the shared information may include properties of the remote virtual network, such as the GRE key of the remote VNET. Also, the information shared between the NRAs 311A-B may include instructions to peer the VNETs A-B through global virtual network peering.
At 604, the shared information, including GRE keys, VNET IDs, etc., is transmitted to agents in the cluster layer 302 from agents in the regional layer 301. For example, the NRAs 311A-B transmit the shared information associated with the VNETs A-B to the respective NCAs 312A-B. This information includes the information for the remote VNET.
At 605, the agents in the cluster layer publish mappings of the customer addresses to the provider addresses, which can be consumed by the agents in the host layer 303. For example, each of NCAs 312A-B determine the mappings for their VNET based on the VNET ID of their VNET, and the mappings are published for the lower layers of the stack, such as NHAs 313A-B, to consume. In an example, the cloud provider may maintain a Virtual Network Directory Service that maintains the mappings from customer IP address space to provider IP address space. Agents may publish the mappings to the Service or subscribe to the mappings from the Service.
At 606, the network management agents, e.g., NHAs 313A-B, at the host layer 303 subscribe to the mappings of the local VNET and the remote VNET. The subscribed mappings may be cached by a host of a VM.
At 607, the subscribed mappings, regional information and/or other information pushed down the network management stack 300, such as IPv6 prefixes, VNET IDs, subnet IDs, etc., from the regional layer to the host layer, are used to peer the VNETs A-B so VMs in the VNETs can communicate directly without going through a virtual network gateway for any of the regions A-B. For example, the VMs 1-2 shown in
For the global virtual network peering of the VNETs A-B, VMs in the VNETs communicate via encapsulated packets that include the regional information and information determined from the mapping subscriptions in the header. An example of the encapsulated packet is shown as 400 in
In an example, a software agent in a host in the host layer 303 may perform encapsulation and decapsulation. For example, referring to
Embodiments and examples are described above, and those skilled in the art will be able to make various modifications to the described embodiments and examples without departing from the scope of the embodiments and examples.