Distributed Resource Scheduling Based on Network Utilization

• US 20180139148A1
• Filed: 11/15/2016
• Published: 05/17/2018
• Est. Priority Date: 11/15/2016
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Gamage, Sahan B., Griffith, Rean
• Granted Patent: US 10,382,352 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06F 9/5088 involving task migration
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/0897 by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
• CPC Class Codes: H04L 41/122 of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/822 Collecting or measuring resource availability data

Abstract

Examples provide a network-utilization based scheduler that considers network utilization when load balancing virtual computing instances (VCIs) in a cluster. The scheduler also periodically evaluates proposed migrations based on a current cluster snapshot including network utilization data of the hosts. Proposed migrations are removed and/or prioritized to prevent network saturation in the cluster.

Claims

1. A computer-implemented method for resource scheduling, the method comprising:

selecting, for placement of a powering-on virtual computing instance (VCI), a candidate set of hosts from a plurality of hosts in a cluster based on processor utilization and memory utilization;

generating a network utilization ranking for each host in the candidate set of hosts based on network utilization associated with each host in the candidate set of hosts;

selecting a host from the candidate set of hosts based on the generated network utilization rankings; and

placing the powering-on VCI on the selected host.

2. The method of claim 1, further comprising:

receiving runtime statistics associated with the plurality of hosts within the cluster via a statistics channel;

generating a current cluster status snapshot based on the received runtime statistics and cluster configuration data; and

generating the network utilization ranking for each host based on network utilization data from the generated current cluster status snapshot.

3. The method of claim 2, further comprising:

on determining a predetermined time interval has elapsed, identifying a proposed migration of a particular VCI from a first host in the cluster to a second host in the cluster;

evaluating the proposed migration based on the current cluster status snapshot;

on determining the second host is network saturated, rejecting the proposed migration; and

on determining the second host is network unsaturated, allowing the proposed migration.

4. The method of claim 1, further comprising:

generating a primary ranking for each host in the plurality of hosts in the cluster, the primary ranking based on processor resources and memory resources associated with each host;

selecting two or more hosts from the plurality of hosts based on the primary ranking, the candidate set of hosts comprising the selected two or more hosts;

generating a secondary ranking for each host in the candidate set of hosts based on processor resources, memory resources, and networking resources; and

selecting the host for placement of the VCI based on the secondary ranking.

5. The method of claim 1, further comprising:

receiving a set of proposed migrations;

generating a network utilization score for each proposed migration in the set of proposed migrations;

comparing each generated network utilization score for each proposed migration to a threshold; and

filtering out each proposed migration associated with a network utilization score exceeding the threshold, wherein the filtered proposed migrations are not performed.

6. The method of claim 1, further comprising:

receiving a set of proposed migrations;

generating a network utilization score for each proposed migration in the set of proposed migrations;

comparing the generated network utilization scores; and

selecting a proposed migration associated with a network utilization score indicating a destination host for the migrating VCI is currently network unsaturated and capable of hosting the VCI without becoming network saturated as a result of the proposed migration.

7. The method of claim 1, further comprising:

comparing the network utilization rankings for each host to a threshold value to identify hosts having a network utilization that exceeds the threshold value; and

in response to identifying a host in the candidate set of hosts having a network utilization that exceeds the threshold value, removing the identified host from the candidate set of hosts.

8. A system for resource scheduling based on network utilization, said system comprising:

at least one memory associated with a set of servers in a cluster, the at least one memory storing a network-utilization based scheduler;

a set of virtual computing instances (VCIs) on the set of servers; and

at least one processor programmed to execute the network-utilization based scheduler to;

generate a set of network utilization scores for each destination host identified in a set of proposed migrations based on a current cluster status snapshot;

compare scores in the generated set of network utilization scores to select a proposed migration from the set of proposed migrations, the selected proposed migration identifying a destination host that is less network saturated than other destination hosts identified in the set of proposed migrations; and

perform the selected proposed migration to move at least one of the VCIs from a first server in the set of servers to a second server in the set of servers.

9. The system of claim 8, further comprising:

a filter component, wherein the at least one processor further executes the filter component to compare a first score in the generated scores to a second score in the generated scores to identify a highest score; and

select a proposed migration associated with the highest score.

10. The system of claim 8, further comprising:

a filter component, wherein the at least one processor further executes the filter component to compare a first score in the generated scores to a threshold score; and

in response to the first score exceeding the threshold score, removing a proposed migration associated with the first score from the set of proposed migrations.

11. The system of claim 10, further comprising selecting another proposed migration.

12. The system of claim 8, further comprising:

on determining a destination host associated with a given proposed migration is network saturated and the VCI has a network load, removing the given proposed migration from the set of proposed migrations; and

on determining a destination host associated with a given proposed migration is network saturated and the VCI to be migrated does not have a network load, approving the proposed migration.

13. The system of claim 8, wherein the current cluster status snapshot comprises host internal network traffic data and external network traffic data, and further comprising:

a set of network utilization counters for collecting the host internal network traffic data and the host external network traffic data; and

a statistics channel for transmitting runtime statistics for the cluster to the network-utilization based scheduler, the runtime statistics including the collected host internal network traffic data and the host external network traffic data.

14. The system of claim 8, wherein the at least one processor is further programmed to execute the network-utilization based scheduler to select the proposed migration from the set of proposed migrations by selecting a host by:

generating a primary ranking for each host in a plurality of hosts based on a set of primary resources;

selecting a candidate set of hosts from the plurality of hosts based on the primary ranking for each host;

generating a secondary ranking for each host in the candidate set of hosts based on network utilization and network availability of each host;

selecting a host having a highest secondary ranking; and

placing the at least one of the VCIs on the selected host.

15. One or more computer storage media embodying computer-executable components, said components comprising:

an initial placement component that is executed to cause at least one processor to rank each host in a candidate set of hosts based on network utilization and selecting a host for placement of a powering-on virtual computing instance (VCI) based on the ranking, the powering-on VCI being placed on the selected host to minimize network saturation of hosts;

a load balancing component that is executed to cause the at least one processor to generate a score for at least one proposed migration in a set of proposed migrations for the powering-on VCI based on network utilization associated with at least one destination host in the set of proposed migrations; and

a filter component that is executed to cause the at least one processor to reject a given proposed migration in the set of proposed migrations in response to a determination that a given score associated with the given proposed migration indicates a destination host associated with the proposed migration is network saturated.

16. The computer storage media of claim 15, wherein a scheduler, upon execution, causes the at least one processor to:

generate a primary ranking for each host in a plurality of hosts in the cluster in response to identifying the powering-on VCI for initial placement, the primary ranking based on host utilization of at least one primary resource;

select a candidate set of hosts from the plurality of hosts based on the generated primary rankings;

generate a secondary ranking of each host in the selected candidate set of hosts based on network utilization; and

place the powering-on VCI on a host selected from the selected candidate set of hosts based on the generated secondary ranking.

17. The computer storage media of claim 16, wherein the at least one primary resource comprises processor resources and memory resources associated with each of the candidate set of hosts, and wherein the secondary ranking is generated based on processor resources, memory resources, and networking resources associated with the candidate set of hosts.

18. The computer storage media of claim 15, wherein the scheduler, upon further execution, causes the at least one processor to:

generate a current cluster status snapshot based on runtime statistics and configuration data, wherein the runtime statistics comprises host internal networking statistics and host external networking statistics; and

generate the primary ranking and the secondary ranking for each host based on the current cluster status snapshot.

19. The computer storage media of claim 15, wherein the filter component, when further executed, causes the at least one processor to:

reject a given proposed migration if the score indicates network saturation associated with the given proposed migration.

20. The computer storage media of claim 15, wherein the filter component, when further executed, causes the at least one processor to:

compare a first score for a first proposed migration to a second score for a second proposed migration; and

remove the first proposed migration from the set of proposed migrations in response to the first score exceeding the second score.