System and method for cooperative virtual machine memory scheduling

• US 8,543,790 B2
• Filed: 01/17/2013
• Issued: 09/24/2013
• Est. Priority Date: 04/27/2006
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Chen, Xiaoxin, Rao, Anil, Waldspruger, Carl A.
• Primary Examiner(s): DUDEK JR, EDWARD J
• Application Number:
  US13/743,853
• Publication Number:
  US 20130145377A1
• Time in Patent Office: 250 Days
• Field of Search: 711/170, 711/E12.002, 718/104
• US Class Current: 711/170
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/1036 for multiple virtual address spaces, e.g. segmentation G06F12/1045 takes precedence
• CPC Class Codes: G06F 12/109 for multiple virtual address spaces, e.g. segmentation G06F12/1036 takes precedence
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2209/508 Monitor
• CPC Class Codes: G06F 2212/151 Emulated environment, e.g. virtual machine
• CPC Class Codes: G06F 9/45545 Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

A resource scheduler for managing a distribution of host physical memory (HPM) among a plurality of virtual machines (VMs) monitors usage by each of the VMs of respective guest physical memories (GPM) to determine how much of the HPM should be allocated to each of the VMs. On determining that an amount of HPM allocated to a source VM should be reallocated to a target VM, the scheduler sends allocation parameters to a balloon application executing in the source VM causing it to reserve and write a value to a guest virtual memory (GVM) location in the source VM. The scheduler identifies the HPM location that corresponds to the reserved GVM and allocates it to the target VM by mapping a guest physical memory location of the target VM to the HPM location.

Claims

1. A method for implementation by a resource scheduler component of a hypervisor, the resource scheduler managing a distribution of host physical memory (HPM) among a plurality of virtual machines (VMs) executing by way of a hypervisor on a host computer system, each VM having guest software, the method comprising:

monitoring usage by each of the VMs of respective guest physical memories to determine how much of the HPM should be allocated to each of the VMs;

on determining that, based on said usage, an amount of HPM allocated to a source VM of the plurality of VMs should be reallocated to a target VM of the plurality of VMs, causing the resource scheduler to;

send allocation parameters to a balloon application executing in the source VM causing the balloon application executing in the source VM to reserve and write a value to a guest virtual memory (GVM) location in the source VM, the GVM location being mapped to a guest physical memory (GPM) location, which is mapped to an HPM location;

identifying the HPM location; and

allocating the HPM location to the target VM by mapping a guest physical memory location of the target VM to the HPM location.

2. The method of claim 1, wherein the identifying of the HPM location comprises receiving the GPM location by way of a message from the balloon application, and converting the GPM location to the HPM location.

3. The method of claim 1, wherein the identifying of the HPM location comprises:

searching HPM locations for the value;

adding HPM locations containing the value to a pool of HPM locations available for allocating to the plurality of VMs; and

selecting the HPM location from the pool of HPM locations for allocation to the target VM.

4. The method of claim 1, wherein the allocating of the HPM location to the target VM comprises assigning the HPM location to a pool of available HPM locations for assigning to ones of the VMs that, according to the monitoring of the VMs, should have more HPM allocated.

5. The method of claim 1, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on relative priorities of VMs.

6. The method of claim 1, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on how much of the respective guest virtual memories of the source VM and the target VM are being actively used by the guest software executing within the source and target VMs.

7. The method of claim 6, wherein the resource scheduler determines that more HPM should be allocated to the target VM when the target VM is actively using a greater percentage of its guest virtual memory than the source VM is using of its guest virtual memory.

8. The method of claim 1, wherein a size of each memory location is one page of memory, wherein the writing of the value to the GVM location includes filling a page of memory corresponding to the GVM location with a pattern.

9. The method of claim 8, wherein the writing of the value further includes filling the page of memory with all zeros.

10. The method of claim 1, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM, additionally marks the GPM location as read only in a pagetable for the source VM so that when the source VM attempts a write to the GPM location, a new HPM location is identified and a mapping is created from the GPM location to the new HPM location.

11. The method of claim 1, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM additionally unmaps the guest physical memory location by setting a page table entry for the GPM location to invalid.

12. A host computer system having a memory system and a processor for executing instructions stored in the memory system, the instructions including a hypervisor that supports execution of a plurality of virtual machines (VMs) each having guest software, they hypervisor including a resource scheduler component for managing a distribution of host physical memory (HPM) among the VMs, the resource scheduler being configured to implement a method including steps of:

monitoring usage by each of the VMs of respective guest physical memories to determine how much of the HPM should be allocated to each of the VMs;

when an amount of HPM allocated to a source VM of the plurality of VMs should be reallocated to a target VM of the plurality of VMs;

sending allocation parameters to a balloon application executing in the source VM causing the balloon application executing in the source VM to reserve and write a value to a guest virtual memory (GVM) location in the source VM, the GVM location being mapped to a guest physical memory (GPM) location, which is mapped to an HPM location;

identifying the HPM location; and

allocating the HPM location to the target VM by mapping a guest physical memory location of the target VM to the HPM location.

13. The host computer system of claim 12, wherein the identifying of the HPM location comprises receiving the GPM location by way of a message from the balloon application, and converting the GPM location to the HPM location.

14. The host computer system of claim 12, wherein the identifying of the HPM location comprises:

searching HPM locations for the value;

adding HPM locations containing the value to a pool of HPM locations available for allocating to the plurality of VMs; and

selecting the HPM location from the pool of HPM locations for allocation to the target VM.

15. The host computer system of claim 12, wherein the allocating of the HPM location to the target VM comprises assigning the HPM location to a pool of available HPM locations for assigning to ones of the VMs that, according to the monitoring of the VMs, should have more HPM allocated.

16. The host computer system of claim 12, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on relative priorities of VMs.

17. The host computer system of claim 12, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on how much of the respective guest virtual memories of the source VM and the target VM are being actively used by the guest software executing within the source and target VMs.

18. The host computer system of claim 17, wherein the resource scheduler determines that more HPM should be allocated to the target VM when the target VM is actively using a greater percentage of its guest virtual memory than the source VM is using of its guest virtual memory.

19. The host computer system of claim 12, wherein a size of each memory location is one page of memory, wherein the writing of the value to the GVM location includes filling a page of memory corresponding to the GVM location with a pattern.

20. The host computer system of claim 19, wherein the writing of the value further includes filling the page of memory with all zeros.

21. The host computer system of claim 12, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM, additionally marks the GPM location as read only in a pagetable for the source VM so that when the source VM attempts a write to the GPM location, a new HPM location is identified and a mapping is created from the GPM location to the new HPM location.

22. The host computer system of claim 12, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM additionally unmaps the guest physical memory location by setting a page table entry for the GPM location to invalid.