Using a hypervisor to provide computer security

• US 7,996,836 B1
• Filed: 12/29/2006
• Issued: 08/09/2011
• Est. Priority Date: 12/29/2006
• Status: Active Grant

• Current Assignee: NortonLifeLock Inc.
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Ferrie, Peter, McCorkendale, Bruce
• Primary Examiner(s): To; Jennifer N
• Application Number:
  US11/618,224
• Time in Patent Office: 1,684 Days
• Field of Search: 718/1
• US Class Current: 718/1
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/57 Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
• CPC Class Codes: G06F 2221/2105 Dual mode as a secondary aspect
• CPC Class Codes: G06F 9/45541 Bare-metal, i.e. hypervisor runs directly on hardware
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

A computer includes a virtual machine controlled by a hypervisor. The virtual machine runs a virtualized operating system with running processes. A security initialization module sets the state in the virtual machine to pass execution from the virtual machine to the hypervisor responsive to a process making a system call in the virtualized operating system. Responsive to execution being passed from the virtual machine to the hypervisor, a security module analyzes the process making the system call to determine whether it poses a security threat. If a security threat is found, the security module takes remedial action to address the threat.

Claims

1. A system for providing security in a computer having a virtual machine controlled by a hypervisor, the virtual machine having an operating system with an operating system kernel and an operating system protection module that detects modifications to the operating system kernel, the system comprising:

a computer processor; and

a non-transitory computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising;

a security initialization module for modifying the operating system kernel of the virtual machine to pass execution from the virtual machine to the hypervisor responsive to a system call issued by a process executing within the virtual machine, wherein modifying the operating system kernel comprises;

setting a breakpoint in the operating system kernel to cause an interrupt upon the system call being issued by the process and

setting an exception bitmap in the virtual machine to pass execution from the virtual machine to the hypervisor responsive to the interrupt;

a disabling module for setting a state in the virtual machine to pass control to the hypervisor during execution of the operating system protection module and for altering functioning of the operating system protection module to prevent the operating system protection module from detecting the modification of the operating system kernel; and

a security module activated responsive to execution being passed to the hypervisor due to the modification by the security initialization module and for analyzing the process to determine whether the process poses a security threat.

2. The system of claim 1, wherein the virtual machine includes a system service dispatch table (SSDT) that contains a reference to the system call and a processor register that provides the location of the SSDT, and wherein the security initialization module uses the processor register to locate the reference to the system call in the SSDT.

3. The system of claim 1, wherein the security module is executed within the hypervisor.

4. The system of claim 1, wherein the security module takes remedial action in response to detecting a security threat.

5. A computer program product having a non-transitory computer-readable medium having computer program instructions recorded thereon for providing security in a computer having a virtual machine controlled by a hypervisor, the virtual machine having an operating system with an operating system kernel and an operating system protection module that detects modifications to the operating system kernel, the computer program product comprising:

a security initialization module for modifying the operating system kernel of the virtual machine to pass execution from the virtual machine to the hypervisor responsive to a system call issued by a process executing within the virtual machine, wherein modifying the operating system kernel comprises;

setting a breakpoint in the operating system kernel to cause an interrupt upon the system call being issued by the process; and

setting an exception bitmap in the virtual machine to pass execution from the virtual machine to the hypervisor responsive to the interrupt;

a disabling module for setting a state in the virtual machine to pass control to the hypervisor during execution of the operating system protection module and for altering functioning of the operating system protection module to prevent the operating system protection module from detecting the modification of the operating system kernel; and

a security module activated responsive to execution being passed to the hypervisor due to the modification by the security initialization module and for analyzing the process to determine whether the process poses a security threat.

6. The computer program product of claim 5, wherein the virtual machine includes a system service dispatch table (SSDT) that contains a reference to the system call and a processor register that provides the location of the SSDT, and wherein the security initialization module uses the processor register to locate the reference to the system call in the SSDT.

7. The computer program product of claim 5, wherein the security module is executed within the hypervisor.

8. The computer program product of claim 5, wherein the security module takes remedial action in response to detecting a security threat.

9. A computer-implemented method of providing security in a computer having a virtual machine controlled by a hypervisor, the virtual machine having an operating system with an operating system kernel and an operating system protection module that detects modifications to the operating system kernel, the method comprising:

modifying an operating system kernel of the virtual machine to pass execution from the virtual machine to the hypervisor responsive to a system call issued by a process executing within the virtual machine, wherein modifying the operating system kernel comprises;

setting a breakpoint in the operating system kernel to cause an interrupt upon the system call being issued by the process; and

setting an exception bitmap in the virtual machine to pass execution from the virtual machine to the hypervisor responsive to the interrupt;

setting a state in the virtual machine to pass control to the hypervisor during execution of the operating system protection module;

responsive to control being passed to the hypervisor due to the state set, altering function of the operating system protection module to prevent the operating system protection module from detecting the modification of the operating system kernel; and

responsive to execution being passed to the hypervisor due to the modification of the operating system kernel, analyzing the process to determine whether the process is a security threat.

10. The computer implemented method of claim 9, wherein the virtual machine includes a system service dispatch table (SSDT) that contains a reference to the system call and a processor register that provides the location of the SSDT, further comprising:

using the processor register to locate the reference to the system call in the SSDT.

11. The computer implemented method of claim 9, wherein analyzing the process to determine whether it is a security threat is executed within the hypervisor.

12. The computer implemented method of claim 9, further comprising:

responsive to the process being determined to be a security threat, taking remedial action.