Controlling network utilization

• US 10,013,281 B2
• Filed: 06/29/2011
• Issued: 07/03/2018
• Est. Priority Date: 06/29/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Fries, Robert
• Primary Examiner(s): Christensen, Scott B
• Assistant Examiner(s): Do, Lam
• Application Number:
  US13/171,445
• Publication Number:
  US 20130007254A1
• Time in Patent Office: 2,561 Days
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2201/815 Virtual
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level

Abstract

Network utilization of an arbitrary application is controlled by tracking network usage statistics of virtual machines (VMs), including at least VMs hosting parts of the application. For network utilization control, VMs serve as network proxies for elements of the application. A specification for a network requirement of the application is evaluated against the network usage statistics. When a network capacity requirement to/from/through an element of the application is not satisfied, one or more VMs are adapted to satisfy the requirement. For example, a VM may be migrated from a host or network location that has excess network bandwidth. Or, for example, network bandwidth availability for an under-requirement VM may be increased and bandwidth availability for a VM at an appropriate host or network location may be decreased. Thus, application-level communication path requirements may be satisfied using VM adaptations.

Claims

1. A method of specifying and allocating network bandwidth for distributed applications executing on virtual machines (VMs) executing on hosts in a network, the method comprising:

storing, by a computing device, application network specifications, each application network specification corresponding to a respective distributed application, each distributed application comprising a plurality of application elements executing on a respective plurality of the VMs on a respective plurality of the hosts, wherein an application network specification of a distributed application comprises individual bandwidth specifications of respective application elements of the distributed application, wherein each bandwidth specification (i) respectively identifies a first application element of the distributed application and a second application element of the distributed application element, and (ii) specifies an inter-element bandwidth condition as a function of a bandwidth requirement between the first application element and the second application element;

collecting, storing, and accessing measures of network usage in the network;

determining, by the computing device, for a given application network specification corresponding to a given distributed application, that an individual given bandwidth specification of the plurality of bandwidth specifications therein is not met, the individual bandwidth specification of the plurality of bandwidth specifications identifying a given first application element of the given network application, a given second application element of the given network application, and a given inter-element bandwidth condition in terms of the given first and second application elements, wherein whether the given bandwidth specification is met is determined by;

selecting, from among the measures of network usage, measures of network bandwidth for respective VMs, wherein the measures of network bandwidth are selected on the basis of the VMs being determined to be hosting instances of the first and second given application elements identified by the given bandwidth specification;

based on the network bandwidth specification, determining an inter-element bandwidth measure corresponding to bandwidth between one or more of the given first application elements and one or more of the given second application elements, wherein the inter-element bandwidth measure is determined using the selected measures of network bandwidth;

determining, by the computing device, that the given inter-element bandwidth condition is not met by evaluating the given inter-element bandwidth condition against the inter-element bandwidth measure; and

based on the determining that the given inter-element bandwidth condition of the individual bandwidth specification of the plurality of bandwidth specifications is not met, selecting one or more VMs on the basis of the selected VMs hosting the given first or second application elements, and migrating the selected one or more of the VMs and/or regulating, by one or more of the hypervisors, bandwidth usage of the selected one or more of the VMs.

2. A method according to claim 1, wherein the application network specification specifies a plurality of communication paths between application elements of the distributed application, and the method further comprises migrating a VM from a first host to a second host to satisfy a network requirement of one of the paths.

3. A method according to claim 1, wherein the measures of network usage include measures computed and provided by hypervisors on the hosts.

4. A method according to claim 3, wherein the measures of network usage include measures computed and reported by one or more routers, switches, or load balancers of the network.

5. A method according to claim 3, wherein at least some of the measures of network usage are obtained by hypervisors generating and inserting probe packets for transmission, receiving the probe packets, measuring a network statistic according to the probe packets, and stripping the probe packets so that they are not passed to a virtual machine.

6. A method according to claim 1, further comprising computing a bandwidth specification dynamically based on a cost minimization function, wherein, according to the cost minimization function, bandwidth cost varies as a function of time.

7. A method of distributing network usage among a plurality of hosts hosting virtual machines (VMs) communicating via a network, wherein each host has a hypervisor that manages the VMs executing thereon, the method performed by a computing device, the method comprising:

accessing network information stored by one or more computers, the network information comprising information specifying network requirements of respective distributed applications running on the VMs, where each distributed application comprises respective plurality of elements that execute on the VMs, and where a network requirement of a given distributed application specifies first and second of application elements of the given distributed application and specifies an intended network performance characteristic of the given distributed application, the network performance characteristic specifying an association between the first application element and the second application element, a statistical function of the association, and a bandwidth condition for the statistical function of the association;

accessing network performance data of the network;

based on the association in the network performance specification, obtaining, from among the performance data, measures of network performance between respective pairs of instances of the first and second application elements, based on the statistical function of the association in the network performance specification, computing the statistical function for the obtained measures of network performance to obtain a network performance statistic, and evaluating the network performance statistic against the bandwidth condition to determine that the network requirement is not satisfied; and

based on the determination that the network requirement is not satisfied, attempting to satisfy the evaluated intended network performance characteristic by (i) migrating VMs among the hosts and/or by (ii) the hypervisors regulating network capacity available to the VMs that the hypervisors manage, wherein the VMs and/or the hypervisors are selected based on the network specification.

8. A method according to claim 7, wherein the network comprises routers, switches, and load balancers, and the guaranteeing further comprises increasing or decreasing network allocations at the routers, switches, or load balancers.

9. A method according to claim 8, further comprising building a model of performance characteristics of the routers, switches, and load balancers, and using the model for the increasing or decreasing of network allocations.

10. A method according to claim 7, further comprising guaranteeing a single bandwidth requirement between one or more of the first application elements instances and one or more of the second application instances.

11. A method according to claim 7, further comprising storing VM network usage statistics of individual VMs and using the VM network usage statistics to select one or more VMs to either be migrated or to have available network capacity increased or decreased.

12. A method according to claim 11, further comprising decreasing network capacity of a VM by suspending the VM.

13. One or more computer-readable storage devices storing information to enable one or more devices to execute a computing device comprising a network controller, the network controller, when executed, performing a process comprising:

accessing stored network statistics data comprising network usage statistics of individual VMs executing on hosts, the network statistics including individual statistics of respective VMs executing first instances of a first element of an application and second instances of a second element of the application;

accessing a network specification for the application, the application comprised of fluctuating numbers of the first and second instances executing on VMs executing on the hosts, the network specification comprising;

a first identifier identifying the first element, a second identifier identifying the second element, an aggregation identifier comprising the first identifier and the second identifier, a bandwidth target, and a Boolean operator specifying a Boolean relationship between the aggregation identifier and the bandwidth target; and

evaluating the network specification against the network statistics to, according to the network specification, identify one or more target VMs from among the VMs executing on the hosts, and initiating VM adaptations for the identified target VMs, the evaluating comprising;

based on the first and second identifiers in the aggregation identifier, obtaining from the network statistics measures of bandwidth between all of the first instances and all the second instances and computing an aggregation of the obtained measures, and

based on the Boolean operator associating the aggregation identifier with the bandwidth target, determining that the computed aggregation and the bandwidth target do not satisfy the Boolean operator.

14. One or more computer-readable storage devices according to claim 13, wherein the VM adaptation comprises an instruction to migrate a VM.

15. One or more computer-readable storage apparatuses according to claim 13, wherein the VM adaptation comprises an instruction for a hypervisor to increase or decrease the network rate or bandwidth of one or more of the target VMs.

16. One or more computer-readable storage devices according to claim 13, wherein the network requirement specifies a statistical calculation for a class or group of the elements.

17. One or more computer-readable storage devices according to claim 13, wherein the network requirement specifies a communication path between an element and a network resource that communicates with the application.

18. One or more computer-readable storage devices according to claim 13, wherein the evaluating comprises identifying a particular VM according to the network specification, selecting a network statistic of the particular VM that was reported by a host hosting the VM, and comparing the network statistic against the network specification.