ONLINE CLASSIFICATION OF MEMORY PAGES BASED ON ACTIVITY LEVEL

• US 20120011504A1
• Filed: 07/12/2010
• Published: 01/12/2012
• Est. Priority Date: 07/12/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: AHMAD, Irfan, WALDSPURGER, Carl A., GARTHWAITE, Alexander Thomas, TATI, Kiran, LU, Pin
• Granted Patent: US 9,032,398 B2
• Time in Patent Office: Days
• US Class Current: 718/1
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/10 Address translation

Abstract

Activity level of memory pages is classified in virtual machine environment, so that processes such as live VM migration and checkpointing, among others, can be carried out more efficiently. The method includes the steps of scanning page table entries of hypervisor-managed page tables continuously over repeating scan periods to determine whether memory pages have been accessed or not, and for each memory page, determining an activity level of the memory page based on whether the memory page has been accessed or not since a prior scan and storing the activity level of the memory page. The activity level of the memory page may be represented by one or more bits of its page table entry and may be classified as having at least two states ranging from hot to cold.

Claims

1. A method of classifying an activity level of memory pages that are accessed by one or more virtual machines executing in a host machine, comprising:

setting a scan rate that defines how frequently page table entries of hypervisor-managed page tables corresponding to the memory pages are to be scanned;

scanning the page table entries over multiple scan periods at the scan rate; and

for each of the memory pages and at each scan period, determining an activity level of the memory page based on whether the memory page has been accessed since a prior scan and storing the activity level of the memory page.

2. The method according to claim 1, wherein the activity level of the memory page is one of high and low.

3. The method according to claim 2, wherein the activity level of the memory page is one of high, low, and at least one intermediate level between high and low.

4. The method according to claim 1, wherein the activity level of the memory page is determined based on whether the memory page has been accessed since a prior scan and a currently stored activity level of the memory page.

5. The method according to claim 4, wherein the stored activity level of the memory page is changed from low to an intermediate level if the memory page has been accessed since a prior scan, and from high to an intermediate level if the memory page has not been accessed since a prior scan.

6. The method according to claim 5, wherein the stored activity level of the memory page is changed from an intermediate level to high if the memory page has been accessed since a prior scan, and from an intermediate level to low if the memory page has not been accessed since a prior scan.

7. The method according to claim 1, wherein the scan rate is set with respect to real time.

8. The method according to claim 1, wherein the memory pages are associated with a virtual machine and the scan rate is set with respect to a virtual execution time of the virtual machine.

9. The method according to claim 1, wherein the scan rate is dynamically adjustable.

10. The method according to claim 1, wherein the page table entries are scanned in order from physical page number 0 to physical page number N-1, where N is the number of memory pages scanned during one scan period.

11. The method according to claim 1, wherein the activity level of a memory page is stored in the page table entry corresponding to the memory page.

12. The method according to claim 1, wherein the page table entries are scanned according to an order defined in a data structure.

13. The method according to claim 12, wherein the data structure includes one or more linked-list data structures.

14. The method according to claim 13, wherein the memory pages having a first activity level have page table entries referenced in a first linked-list data structure and the memory pages having a second activity level have page table entries referenced in a second linked-list data structure.

15. The method according to claim 12, wherein the data structure includes a linked-list data structure, and each entry in the linked-list data structure corresponds to a memory page and includes an indication of the currently stored activity level of the memory page and a pointer to the next page table entry to be scanned.

16. The method according to claim 1, wherein the activity level of a memory page is determined by examining whether the accessed bit of the page table entry corresponding to the memory page is set.

17. The method according to claim 16, further comprising:

clearing the accessed bit of the page table entry corresponding to the memory page after determining the activity level of the memory page.

18. The method according to claim 17, further comprising:

clearing any cached state of the accessed bit of the page table entry corresponding to the memory page after determining the activity level of the memory page.

19. The method according to claim 1, wherein the activity level of a memory page is determined by examining whether the dirty bit of the page table entry corresponding to the memory page is set.

20. The method according to claim 19, further comprising:

clearing the dirty bit of the page table entry corresponding to the memory page after determining the activity level of the memory page.

21. The method according to claim 20, further comprising:

clearing any cached state of the dirty bit of the page table entry corresponding to the memory page after determining the activity level of the memory page.

22. The method according to claim 1, wherein the activity level of the memory page is determined based on whether the memory page has been read from or written to since a prior scan.

23. The method according to claim 1, wherein the activity level of the memory page is determined based on whether the memory page has been written to since a prior scan.

24. The method according to claim 1, wherein functions of scanning the page table entries over multiple scan periods at the scan rate and determining, for each of the memory pages and at each scan period, an activity level of the memory page based on whether the memory page has been accessed since a prior scan, are implemented in hardware, and said scanning and said determining are performed by said hardware.

25. A computer system including one or more virtual machines running therein, and a hypervisor for managing page tables that provide mappings from guest physical memory pages to machine memory pages, wherein the hypervisor is programmed to scan entries of the pages tables over repeating scan periods in accordance with a scan rate and determine an activity level of a machine memory page corresponding to the scanned page table entry based on whether an access or dirty bit of the scanned page table entry is set and store the activity level of the machine memory page.

26. The computer system of claim 25, wherein the hypervisor is programmed with a finite state machine for determining the activity level of the machine memory page.

27. The computer system of claim 26, wherein the activity level of the machine memory page is one of a low state, a high state, and first, second, and third intermediate states, and the finite state machine transitions a current state of a machine memory page to a new state based on whether an access or dirty bit of a scanned page table entry corresponding to the machine memory page is set.

28. The computer system of claim 26, wherein if the access or dirty bit is set, the state transitions include low to second intermediate, first intermediate to high, second intermediate to third intermediate, and third intermediate to high, and if the access or dirty bit is not set, the state transitions include high to third intermediate, third intermediate to first intermediate, second intermediate to first intermediate, and first intermediate to low.

29. A non-transitory computer-readable storage medium comprising instructions, which when executed in a computer system including a hypervisor for managing page tables that provide mappings from guest physical memory pages to machine memory pages, causes the computer system to carry out the steps of:

scanning page table entries of hypervisor-managed page tables over multiple scan periods according to a pre-defined scan rate; and

for each of the memory pages and at each scan period, determining an activity level of the memory page based on whether the memory page has been accessed since a prior scan and storing the activity level of the memory page.

30. The non-transitory computer-readable storage medium of claim 29, wherein the activity level of the memory page is determined based on whether the memory page has been accessed since a prior scan and a currently stored activity level of the memory page.

31. The non-transitory computer-readable storage medium of claim 30, wherein the activity level of the memory page is one of high and low.

32. The non-transitory computer-readable storage medium of claim 30, wherein the activity level of the memory page is one of high, low, and at least one intermediate level between high and low.

33. The non-transitory computer-readable storage medium of claim 29, wherein the activity level of a memory page is stored in the page table entry corresponding to the memory page.

34. The non-transitory computer-readable storage medium of claim 29, wherein the memory pages having a first activity level are referenced in a first data structure and the memory pages having a second activity level are referenced in a second data structure, and the step of storing the activity level of the memory page includes moving the reference of the memory page between the first and second data structures.

35. A method of classifying an activity level of memory pages that are accessed by one or more virtual machines executing in a host machine, comprising:

setting a frequency for checking access to the memory pages; and

for each of the memory pages and each time the memory page is checked for access, determining an activity level of the memory page based on how frequently the memory page has been accessed and storing the activity level of the memory page.

36. The method according to claim 35, wherein the activity level of the memory page is one of high and low.

37. The method according to claim 35, wherein the activity level of the memory page is one of high, low, and at least one intermediate level between high and low.

38. The method according to claim 35, wherein a hypervisor performs the steps of setting, determining, and storing.