System and method for critical address space protection in a hypervisor environment

• US 8,694,738 B2
• Filed: 10/11/2011
• Issued: 04/08/2014
• Est. Priority Date: 10/11/2011
• Status: Active Grant

• Current Assignee: McAfee, LLC
• Original Assignee: McAfee, Inc. (McAfee, LLC)
• Inventors: Bhattacharjee, Rajbir, Munjal, Nitin, Singh, Balbir, Singh, Pankaj
• Primary Examiner(s): Elmore, Reba I
• Application Number:
  US13/271,102
• Publication Number:
  US 20130091318A1
• Time in Patent Office: 910 Days
• Field of Search: 711/6, 711/151, 711/163
• US Class Current: 711/151
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 12/145 the protection being virtual, e.g. for virtual blocks or segments before a translation mechanism
• CPC Class Codes: G06F 12/1458 by checking the subject access rights
• CPC Class Codes: G06F 12/1475 in a virtual system, e.g. with translation means
• CPC Class Codes: G06F 12/1491 in a hierarchical protection system, e.g. privilege levels, memory rings
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A system and method in one embodiment includes modules for detecting an access attempt to a critical address space (CAS) of a guest operating system (OS) that has implemented address space layout randomization in a hypervisor environment, identifying a process attempting the access, and taking an action if the process is not permitted to access the CAS. The action can be selected from: reporting the access to a management console of the hypervisor, providing a recommendation to the guest OS, and automatically taking an action within the guest OS. Other embodiments include identifying a machine address corresponding to the CAS by forcing a page fault in the guest OS, resolving a guest physical address from a guest virtual address corresponding to the CAS, and mapping the machine address to the guest physical address.

Claims

1. A method comprising:

detecting an access attempt to a critical address space (CAS) of a guest operating system (OS) in a hypervisor environment comprising a hypervisor, wherein address space layout randomization (ASLR) is implemented by the guest OS;

identifying a process attempting the access; and

taking an action if the process is not permitted to access the CAS.

2. The method of claim 1, wherein the detecting the access attempt comprises:

generating page table entries (PTEs) for pages corresponding to the CAS in a shadow page table of the hypervisor;

marking the PTEs such that the access attempt results in a page fault.

3. The method of claim 1, wherein the identifying the process attempting the access comprises reading a CR3 register corresponding to the process.

4. The method of claim 1, wherein the action is selected from:

reporting the access to a management console of the hypervisor;

providing a recommendation to the guest OS; and

automatically taking an action within the guest OS.

5. The method of claim 4, wherein the reporting the access to a management console of the hypervisor includes flagging a status of the guest OS as infected.

6. The method of claim 4, wherein the providing a recommendation to the guest OS comprises at least one of:

recommending that the process be blacklisted, until it is scanned and whitelisted by a security tool;

or

running an anti-virus on the process.

7. The method of claim 4, wherein the taking an action within the guest OS comprises at least one of:

running an anti-virus program in the guest OS;

or

shutting down or saving a state of the guest OS for offline scanning.

8. The method of claim 1, further comprising validating the access attempt using a policy, comprising:

denying the access if the process is executing from a writeable area of a memory element; and

permitting the access if the process is executing from a read-only area of the memory element.

9. The method of claim 1, further comprising:

identifying a machine address corresponding to the CAS, the identifying the machine address comprising;

forcing a page fault in the guest OS;

resolving a guest physical address from a guest virtual address corresponding to the CAS; and

mapping the machine address to the guest physical address.

10. An apparatus comprising:

a memory element configured to store data; and

a computing processor operable to execute instructions associated with the data;

a hypervisor; and

an agent residing in a guest operating system (OS), such that the apparatus is configured for;

detecting an access attempt to a critical address space (CAS) of the guest OS in a hypervisor environment comprising the hypervisor, wherein address space layout randomization (ASLR) is implemented by the guest OS;

identifying a process attempting the access; and

taking an action if the process is not permitted to access the CAS.

11. The apparatus of claim 10, wherein the detecting the access attempt comprises:

generating page table entries (PTEs) for pages corresponding to the CAS in a shadow page table of the hypervisor;

marking the PTEs such that the access attempt results in a page fault.

12. The apparatus of claim 10, wherein the identifying the process attempting the access comprises reading a CR3 register corresponding to the process.

13. The apparatus of claim 10, wherein the action is selected from:

reporting the access to a management console of the hypervisor;

providing a recommendation to the guest OS; and

automatically taking an action within the guest OS.

14. The apparatus of claim 10, wherein the apparatus is further configured for:

identifying a machine address corresponding to the CAS, the identifying comprising;

forcing a page fault in the guest OS;

resolving a guest physical address from a guest virtual address corresponding to the CAS; and

mapping the machine address to the guest physical address.

15. Logic encoded in non-transitory media that includes code for execution and when executed by a processor is operable to perform operations comprising:

detecting an access attempt to a critical address space (CAS) of a guest operating system (OS) in a hypervisor environment comprising a hypervisor, wherein address space layout randomization (ASLR) is implemented by the guest OS;

identifying a process attempting the access; and

taking an action if the process is not permitted to access the CAS.

16. The logic of claim 15, wherein the detecting the access attempt comprises:

generating page table entries (PTEs) for pages corresponding to the CAS in a shadow page table of the hypervisor;

marking the PTEs such that the access attempt results in a page fault.

17. The logic of claim 15, wherein the identifying the process attempting the access comprises reading a CR3 register corresponding to the process.

18. The logic of claim 15, wherein the action is selected from:

reporting the access to a management console of the hypervisor;

providing a recommendation to the guest OS; and

automatically taking an action within the guest OS.

19. The logic of claim 15, further comprising validating the access attempt using a policy, comprising:

denying the access if the process is executing from a writeable area of a memory element; and

permitting the access if the process is executing from a read-only area of the memory element.

20. The logic of claim 15, further comprising:

identifying a machine address corresponding to the CAS, the identifying comprising;

forcing a page fault in the guest OS;

resolving a guest physical address from a guest virtual address corresponding to the CAS; and

mapping the machine address to the guest physical address.