Management of Supervisor Mode Execution Protection (SMEP) by a Hypervisor

• US 20140351810A1
• Filed: 05/24/2013
• Published: 11/27/2014
• Est. Priority Date: 05/24/2013
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Bromium Inc (HP Inc.)
• Inventors: Pratt, Ian, Wojtczuk, Rafal
• Granted Patent: US 9,292,328 B2
• Time in Patent Office: Days
• US Class Current: 718/1
• CPC Class Codes: G06F 9/45545 Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox

Abstract

Approaches for enabling Supervisor Mode Execution Protection (SMEP) for a guest operating system which does not support SMEP. A guest operating system (OS), which does not support SMEP, is executed within a virtual machine. A hypervisor instructs hardware to enable SMEP for the virtual machine executing the guest operating system. When the hypervisor is notified that the hardware has detected the guest operating system instructing a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor may consult a policy to identify what, if any, responsive action the hypervisor should perform.

Claims

1. One or more computer-readable storage mediums storing one or more sequences of instructions, which when executed by one or more processors, cause:

executing, within a virtual machine, a guest operating system which does not support Supervisor Mode Execution Protection (SMEP); and

a hypervisor instructing hardware to enable Supervisor Mode Execution Protection (SMEP) for the virtual machine executing the guest operating system.

2. The one or more computer-readable storage mediums of claim 1, wherein the guest operating system is Microsoft Windows 7.

3. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor consulting a policy to identify what, if any, responsive action the hypervisor should perform.

4. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

upon the hardware detecting that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor allowing a page fault to propagate to the guest operating system to cause the guest operating system to crash.

5. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

upon the hardware detecting that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor allowing a page fault to propagate to the guest operating system to cause the guest operating system to loop attempting to execute the faulting instruction.

6. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

the hypervisor determining that a SMEP page fault has been propagated to the guest operating system upon observing that the guest operating system is repeatedly attempting to execute a faulting instruction.

7. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor terminating the virtual machine.

8. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor (a) suppressing the page fault to prevent the page fault from being propagated to the guest operating system, (b) the hypervisor instructing the hardware to disable Supervisor Mode Execution Protection (SMEP) for the guest operating system, and (c) continuing execution of the guest operating system and initiating forensic analysis upon activity performed within the virtual machine.

9. The one or more computer-readable storage mediums of claim 1, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor (a) suppressing the page fault to prevent the page fault from being propagated to the guest operating system, (b) the hypervisor emulating a successful performance of the instruction which caused the central processing unit (CPU) to execute code stored in memory designated as user space while the CPU is in supervisor mode, and (c) initiating forensic analysis upon activity performed within the virtual machines.

10. An apparatus, comprising:

one or more processors; and

one or more computer-readable storage mediums storing one or more sequences of instructions, which when executed by the one or more processors, cause;

executing, within a virtual machine, a guest operating system which does not support Supervisor Mode Execution Protection (SMEP); and

a hypervisor instructing hardware to enable Supervisor Mode Execution Protection (SMEP) for the virtual machine executing the guest operating system.

11. The apparatus of claim 10, wherein the guest operating system is Microsoft Windows 7.

12. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor consulting a policy to identify what, if any, responsive action the hypervisor should perform.

13. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

upon the hardware detecting that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor allowing a page fault to be propagated to the guest operating system to cause the guest operating system to crash.

14. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

upon the hardware detecting that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor allowing a page fault to be propagated to the guest operating system to cause the guest operating system to loop attempting to execute the faulting instruction.

15. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

the hypervisor determining that a SMEP page fault has been propagated to the guest operating system upon observing that the guest operating system is repeatedly attempting to execute a faulting instruction.

16. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor terminating the virtual machine.

17. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor (a) suppressing the page fault to prevent the page fault from being propagated to the guest operating system, (b) the hypervisor instructing the hardware to disable Supervisor Mode Execution Protection (SMEP) for the guest operating system, and (c) continuing execution of the guest operating system and initiating forensic analysis upon activity performed within the virtual machine.

18. The apparatus of claim 10, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor (a) suppressing the page fault to prevent the page fault from being propagated to the guest operating system, (b) the hypervisor emulating a successful performance of the instruction which caused the central processing unit (CPU) to execute code stored in memory designated as user space while the CPU is in supervisor mode, and (c) initiating forensic analysis upon activity performed within the virtual machines.

19. A method, comprising:

executing, within a virtual machine, a guest operating system which does not support Supervisor Mode Execution Protection (SMEP); and

a hypervisor instructing hardware to enable Supervisor Mode Execution Protection (SMEP) for the virtual machine executing the guest operating system.

20. The method of claim 19, wherein the guest operating system is Microsoft Windows 7.

21. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor consulting a policy to identify what, if any, responsive action the hypervisor should perform.

22. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

upon the hardware detecting that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor allowing a page fault to be propagated to the guest operating system to cause the guest operating system to crash.

23. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

upon the hardware detecting that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor allowing a page fault to be propagated to the guest operating system to cause the guest operating system to loop attempting to execute the faulting instruction.

24. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

the hypervisor determining that a SMEP page fault has been propagated to the guest operating system upon observing that the guest operating system is repeatedly attempting to execute a faulting instruction.

25. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor terminating the virtual machine.

26. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor (a) suppressing the page fault to prevent the page fault from being propagated to the guest operating system, (b) the hypervisor instructing the hardware to disable Supervisor Mode Execution Protection (SMEP) for the guest operating system, and (c) continuing execution of the guest operating system and initiating forensic analysis upon activity performed within the virtual machine.

27. The method of claim 19, wherein execution of the one or more sequences of instructions further causes:

upon the hypervisor being notified that the hardware detected that the guest operating system has instructed a central processing unit (CPU) to execute code stored in virtual memory accessible by user space while the CPU is in supervisor mode, the hypervisor (a) suppressing the page fault to prevent the page fault from being propagated to the guest operating system, (b) the hypervisor emulating a successful performance of the instruction which caused the central processing unit (CPU) to execute code stored in memory designated as user space while the CPU is in supervisor mode, and (c) initiating forensic analysis upon activity performed within the virtual machines.