Technique for protecting guest processes using a layered virtualization architecture

• US 10,447,728 B1
• Filed: 08/05/2016
• Issued: 10/15/2019
• Est. Priority Date: 12/10/2015
• Status: Active Grant

• Current Assignee: Fireeye Security Holdings US LLC
• Original Assignee: FireEye, Inc.
• Inventors: Steinberg, Udo
• Primary Examiner(s): Gee, Jason K
• Assistant Examiner(s): Cattungal, Dereena T
• Application Number:
  US15/230,215
• Time in Patent Office: 1,166 Days
• Field of Search: 726 1
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 12/109 for multiple virtual address spaces, e.g. segmentation G06F12/1036 takes precedence
• CPC Class Codes: G06F 12/145 the protection being virtual, e.g. for virtual blocks or segments before a translation mechanism
• CPC Class Codes: G06F 12/1475 in a virtual system, e.g. with translation means
• CPC Class Codes: G06F 16/282 Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
• CPC Class Codes: G06F 16/9017 using directory or table look-up use of a directory or look-up table in file systems G06F16/13
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 2212/151 Emulated environment, e.g. virtual machine
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/657 Virtual address space management
• CPC Class Codes: G06F 2221/2113 Multi-level security, e.g. mandatory access control
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/145 : the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
• CPC Class Codes: H04L 63/20 : for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

A technique protects guest processes of a guest operating system kernel using a virtualization layer of a virtualization architecture executing on a node of a network environment. The virtualization layer may include a user mode portion having hyper-processes and a kernel portion having an micro-hypervisor that cooperate to virtualize the guest operating system kernel within a virtual machine and to make hardware resources of the node available for use by the guest operating system kernel, either as pass-through resources, emulated resources, or a combination thereof. Illustratively, the micro-hypervisor may cooperate with the hyper-processes of the virtualization layer to protect the guest processes against attack by one or more exploits that may employ malware. To that end, the guest process protection technique enables the micro-hypervisor and/or hyper-processes of the virtualization layer to determine (i) when the guest operating system switches to a guest process for execution, (ii) an identity of the guest process, and (iii) a protection policy to be associated with the guest process identity.

Claims

1. A system comprising:

a memory configured to store a guest process, a guest operating system kernel and a virtualization layer;

a memory management unit (MMU) coupled to and separate from the memory, the MMU including a guest page table hierarchy associated with the guest process; and

a central processing unit (CPU) coupled to the MMU and adapted to execute the guest process, the guest operating system kernel and the virtualization layer, the CPU including a control register represented by a virtual control register within the virtualization layer, the virtualization layer, when executed, being operable to;

determine that the guest operating system kernel is switching from a prior guest process to the guest process for execution on the CPU when an address for the guest page table hierarchy associated with the guest process is loaded into the virtual control register;

determine an identity of the guest process associated with the guest page table hierarchy using at least one of an agent of the guest operating system, guest operating system specific knowledge, or content analysis of the guest process; and

apply a protection profile associated with the identified guest process to override permissions of one or more memory pages of the guest process as defined by the guest page table hierarchy.

2. The system of claim 1 wherein the guest process runs in an address space of guest-virtual addresses and has one or more guest page tables of the guest page table hierarchy associated with the address space.

3. The system of claim 2 wherein, during a context switch when the guest operating system kernel switches from the prior guest process to the guest process, the guest operating system kernel changes a prior guest page table hierarchy associated with the prior guest process to the guest page table hierarchy of the guest process by loading the address of the guest page table hierarchy into the virtual control register.

4. The system of claim 1 wherein each of the prior guest process and the guest process runs in a separate guest address space and one or more guest page tables form the guest page table hierarchy.

5. The system of claim 1 wherein the virtualization layer when executed is further operable to communicate with the agent of the guest operating system to acquire the identity of the guest process from a guest operating system data structure.

6. The system of claim 1 wherein the virtualization layer when executed is further operable to use the guest operating system specific knowledge to parse a guest operating system data structure to reveal a field having a path that includes a name for the guest process and to deduce the identity of the guest process from the process name.

7. The system of claim 1 wherein the virtualization layer when executed is further operable to perform the content analysis of the guest process by hashing one or more code pages of the guest process marked by the guest page table hierarchy to identify the guest process.

8. The system of claim 1 wherein the guest page table hierarchy is controlled by the guest operating system kernel to translate a guest-virtual address to a guest-physical address.

9. The system of claim 8 wherein the MMU includes a nested page table hierarchy that is controlled by the virtualization layer to translate the guest-physical address to a host-physical address used to access the memory.

10. The system of claim 9 wherein the virtualization layer when executed is further operable to override the permissions of the one or more memory pages of the guest process as defined by the guest page table hierarchy using the guest-physical address to host-physical address translation performed at the nested page table hierarchy on a per process basis.

11. The system of claim 1 wherein the protection profile contains information about process events to be intercepted and specifies restrictions to be applied when the guest process is active.

12. The system of claim 11 wherein the protection profile applies different restrictions for different instances of the guest process.

13. The system of claim 11 wherein the protection profile applies similar restrictions for a plurality of instances of the guest process.

14. The system of claim 1 wherein the virtualization layer including a user mode portion including one or more hyper-processes and a kernel portion including at least a micro-hypervisor that cooperate to virtualize the guest operating system kernel within the virtual machine and to make hardware resources of the system available for use by the guest operating system kernel.

15. The system of claim 1, wherein the agent of the guest operating system is a protected component within the guest operating system communicatively coupled to a kernel portion of the virtualization layer over an interface.

16. The method of claim 15 wherein the interface is a privileged interface embodied as a set of defined hyper-calls, each of the hyper-calls to communicate with a micro-hypervisor deployed within the kernel portion of the virtualization layer.

22. The method of claim 14 wherein the determining of the identity of the guest process associated with the guest page table hierarchy at the virtualization layer is conducted using one of an agent of the guest operating system, guest operating system specific knowledge, and content analysis of the guest process.

17. A method comprising:

storing one or more guest page tables of a guest page table hierarchy and one or more nested page tables of a nested page table hierarchy;

storing an address for the guest page hierarchy in a virtual control register being part of a virtualization layer and corresponding to a control register of a central processing unit (CPU) of a node, the CPU being adapted to execute a guest process, a guest operating system kernel and the virtualization layer resident in a memory of the node;

determining that the guest operating system kernel switches from a prior guest process to the guest process for execution on the CPU when the address for the guest page table hierarchy associated with the guest process is loaded into the virtual control register;

determining an identity of the guest process associated with the guest page table hierarchy at the virtualization layer using at least one of an agent of the guest operating system, guest operating system specific knowledge, or content analysis or the guest process; and

applying a protection profile associated with the identified guest process at the virtualization layer to override permissions of one or more code pages of the guest process as defined by the guest page table hierarchy.

18. The method of claim 17, wherein the determining that the guest operating system kernel switches from the prior guest process associated with the prior guest page table hierarchy to the guest process associated with the guest page table hierarchy includes intercepting a write access from the guest operating system kernel to the virtual control register.

19. The method of claim 17 wherein using the agent of the guest operating system to determine the identity of the guest process associated with the guest page table hierarchy at the virtualization layer comprises communicating with the agent to acquire the identity of the guest process from a guest operating system data structure when the guest process is created.

20. The method of claim 17 wherein wherein using the guest operating system specific knowledge to determine the identity of the guest process associated with the guest page table hierarchy at the virtualization layer comprises parsing a guest operating system data structure to reveal a field having a path that includes a name for the guest process and deducing the identity of the guest process from the process name.

21. The method of claim 17 wherein using the content analysis of the guest process to determine the identity of the guest process associated with the guest page table hierarchy at the virtualization layer comprises hashing the code pages of the guest process marker by the guest page table hierarchy to identify the guest process.

23. The method of claim 17 wherein the virtualization layer including a user mode portion including one or more hyper-processes and a kernel portion including at least a micro-hypervisor that cooperate to virtualize the guest operating system kernel within the virtual machine and to make hardware resources of the system available for use by the guest operating system kernel.

24. The method of claim 17, wherein the agent of the guest operating system is a protected component within the guest operating system communicatively coupled to a kernel portion of the virtualization layer over an interface.

25. The method of claim 24, wherein the interface is a privileged interface embodied as a set of defined hyper-calls, each of the hyper-calls to communicate with a micro-hypervisor deployed within the kernel portion of the virtualization layer.

26. A non-transitory computer readable media containing instructions for execution on a central processing unit (CPU) of a node that performs operations comprising:

storing one or more guest page tables of a guest page table hierarchy and one or more nested page tables of a nested page table hierarchy;

storing an address for the guest page table hierarchy in a virtual control register associated with the CPU, the CPU being adapted to execute a guest process, a guest operating system kernel and a virtualization layer resident in a memory of the node;

determining that the guest operating system kernel switches from a prior guest process to the guest process for execution on the CPU when the address for the guest page table hierarchy associated with the guest process is loaded into the virtual control register;

determining an identity of the guest process associated with the guest page table hierarchy at the virtualization layer using at least one of an agent of the guest operating system, guest operating system specific knowledge, or content analysis of the guest process; and

applying a protection profile associated with the identified guest process at the virtualization layer to override permissions of one or more memory pages of the guest process as defined by the guest page table hierarchy.

27. The non-transitory computer readable media of claim 26 wherein the virtualization layer includes a user mode portion including one or more hyper-processes and a kernel portion including at least a micro-hypervisor that cooperate to virtualize the guest operating system kernel within the virtual machine and to make hardware resources of the node available for use by the guest operating system.

28. The non-transitory computer readable media of claim 26 wherein the using of the agent of the guest operating system to determine the identity of the guest process associated with the guest page table hierarchy at the virtualization layer comprises communication with the agent to acquire the identity of the guest process from a guest operating system data structure when the guest process is created.

29. The non-transitory computer readable media of claim 26 wherein the using of the guest operating system specific knowledge to determine the identity of the guest process associated with the guest page table hierarchy at the virtualization layer comprises parsing a guest operating system data structure to reveal a field having a path that includes a name for the guest process and deducing the identity of the guest process from the process name.

30. The non-transitory computer readable media of claim 26 wherein the using of the content analysis of the guest process to determine the identity of the guest process associated with the guest page table hierarchy at the virtualization layer comprises hashing the code pages of the guest process marked by the guest page table hierarchy to identify the guest process.

31. The non-transitory computer readable media of claim 26 wherein the applying of the protection profile associated with the identified guest process at the virtualization layer comprises overriding the permissions of the one or more code pages using a guest-physical address to host-physical address translation performed at the nest page table hierarchy.

32. The non-transitory computer readable media of claim 26, wherein the instructions for execution performs the operation of determining that the guest operating system kernel switches from the prior guest process to the guest process comprises intercepting a write access from the guest operating system kernel to the virtual control register.