Secure public cloud with protected guest-verified host control

• US 10,303,899 B2
• Filed: 02/28/2017
• Issued: 05/28/2019
• Est. Priority Date: 08/11/2016
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Durham, David M., Neiger, Gilbert, Huntley, Barry E., Sahita, Ravi L., Patel, Baiju V.
• Primary Examiner(s): Dada, Beemnet W
• Application Number:
  US15/444,771
• Publication Number:
  US 20180247082A1
• Time in Patent Office: 819 Days
• CPC Class Codes: G06F 12/1441 for a range
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/57 Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
• CPC Class Codes: G06F 21/71 to assure secure computing or processing of information
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 2212/402 Encrypted data
• CPC Class Codes: G06F 2221/2149 Restricted operating environment
• CPC Class Codes: G06F 8/63 Image based installation; Cloning; Build to order
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 9/0822 using key encryption key

Abstract

A host Virtual Machine Monitor (VMM) operates “blindly,” without the host VMM having the ability to access data within a guest virtual machine (VM) or the ability to access directly control structures that control execution flow of the guest VM. Guest VMs execute within a protected region of memory (called a key domain) that even the host VMM cannot access. Virtualization data structures that pertain to the execution state (e.g., a Virtual Machine Control Structure (VMCS)) and memory mappings (e.g., Extended Page Tables (EPTs)) of the guest VM are also located in the protected memory region and are also encrypted with the key domain key. The host VMM and other guest VMs, which do not possess the key domain key for other key domains, cannot directly modify these control structures nor access the protected memory region. The host VMM, however, can verify correctness of the control structures of guest VMs.

Claims

1. An apparatus comprising:

a processor; and

a memory coupled to the processor;

wherein

the processor is to execute an untrusted host virtual machine monitor to manage execution by the processor of at least one guest virtual machine;

the untrusted host virtual machine monitor is to receive an encrypted key domain key, an encrypted guest code image encrypted by a key domain key, and an encrypted guest control structure encrypted by the key domain key, the key domain key inaccessible to the untrusted host virtual machine monitor;

the untrusted host virtual machine monitor is to issue a create command to the processor to create a first key domain, the first key domain comprising a region of the memory to be encrypted by the key domain key;

in response to receiving the create command, the processor is to create the first key domain and decrypt the encrypted key domain key to produce the key domain key;

the untrusted host virtual machine monitor is to issue a launch command to the processor to launch a first guest virtual machine within the first key domain;

in response to receiving the launch command, the processor is to (a) switch to the first key domain, (b) decrypt the encrypted guest control structure to produce a guest control structure containing guest processor state information, (c) decrypt the encrypted guest code image to produce a guest code image, and (d) execute the guest code image within the first key domain using the guest processor state information, wherein the guest control structure specifies a protected location of the memory where the processor may store the guest processor state information;

in response to an event triggering an exit condition of the first guest virtual machine, the processor is to save the guest processor state information for the first guest virtual machine in the protected location of the memory;

the untrusted host virtual machine monitor is to issue a resume command to the processor to resume the first guest virtual machine; and

in response to receiving the resume command, the processor is to (a) switch to the first key domain, (b) retrieve the guest processor state information for the first guest virtual machine from the protected location of the memory, and (c) execute the guest code image within the first key domain using the guest processor state information.

2. The apparatus of claim 1, wherein

in response to the event triggering the exit condition of the first guest virtual machine, the processor is to switch from the first key domain to a second key domain.

3. The apparatus of claim 1, wherein

the untrusted host virtual machine monitor is to verify the encrypted guest control structure.

4. The apparatus of claim 1, wherein

the guest code image comprises interrupt handler code to intercept an interrupt;

the processor is to convert the exit condition of the first guest virtual machine to an exception;

the guest code image is to save processor register information to the protected location of the memory in response to at least one of the interrupt and the exception;

the guest code image is to clear a first processor register if the first processor register is not needed by the untrusted host virtual machine monitor;

the guest code image is to conditionally expose a second processor register if the second processor register is needed by the untrusted host virtual machine monitor;

the guest code image is to invoke the untrusted host virtual machine monitor; and

the first guest virtual machine is to exit when the untrusted host virtual machine monitor is invoked.

5. The apparatus of claim 1, wherein

the untrusted host virtual machine monitor is to receive an encrypted updated guest control structure, install the encrypted updated guest control structure in the memory, and verify the encrypted updated guest control structure;

the processor is to decrypt the encrypted updated guest control structure to produce an updated guest control structure;

in response to verifying the encrypted updated guest control structure, the untrusted host virtual machine monitor is to issue an enter command to the processor to enter the first guest virtual machine using the updated guest control structure; and

in response to receiving the enter command, the processor is to enter the first guest virtual machine using the updated guest control structure.

6. The apparatus of claim 5, wherein

the untrusted host virtual machine monitor is further to receive an encrypted updated guest code image and to install the encrypted updated guest code image in the memory;

the processor is to decrypt the encrypted updated guest code image update to produce an updated guest code image; and

the processor is to enter the first guest virtual machine by executing the updated guest code image using the updated guest control structure.

7. The apparatus of claim 1, wherein

the untrusted host virtual machine monitor is to determine whether a change to the guest control structure is needed;

the first guest virtual machine is to verify that the change to the guest control structure does not compromise security of the first guest virtual machine;

the first guest virtual machine is to produce an encrypted updated guest control structure incorporating the change using the key domain key; and

the first guest virtual machine is to send the encrypted updated guest control structure to the untrusted host virtual machine monitor via a shared region of memory shared by the untrusted host virtual machine monitor and the first guest virtual machine.

8. The apparatus of claim 7, wherein

the untrusted host virtual machine monitor is to install the encrypted updated guest control structure in the memory;

the untrusted host virtual machine monitor is to verify the encrypted updated guest control structure;

the processor is to decrypt the encrypted updated guest control structure to produce an updated guest control structure;

in response to verifying the encrypted updated guest control structure, the untrusted host virtual machine monitor is to issue an enter command to the processor to enter the first guest virtual machine using the updated guest control structure; and

in response to receiving the enter command, the processor is to enter the first guest virtual machine using the updated guest control structure.

9. The apparatus of claim 1, wherein

the encrypted guest code image comprises an agent code image;

the encrypted guest control structure comprises an agent control structure;

the untrusted host virtual machine monitor is to verify the agent control structure;

the untrusted host virtual machine monitor is to issue a second launch command to the processor to launch a second guest virtual machine within the first key domain, the second guest virtual machine to provide an agent to act on behalf of the untrusted host virtual machine monitor within the first key domain; and

in response to receiving the second launch command, the processor is to switch to the first key domain, decrypt the encrypted guest code image to produce the agent code image, decrypt the encrypted guest control structure to produce the agent control structure containing agent processor state information, and execute the agent code image within the first key domain using the agent processor state information.

10. The apparatus of claim 9, wherein

the untrusted host virtual machine monitor is to communicate a request to modify the guest control structure of the first guest virtual machine to the agent via a shared region of memory shared by the agent and the untrusted host virtual machine monitor;

in response to reading the request from the shared region of memory, the agent is to modify the guest control structure of the first guest virtual machine within the first key domain to produce a modified guest control structure of the first guest virtual machine;

the untrusted host virtual machine monitor is to verify the modified guest control structure of the first guest virtual machine;

upon verifying the modified guest control structure, the untrusted host virtual machine monitor is to issue an enter command to the processor to enter the first guest virtual machine within the first key domain; and

in response to receiving the enter command, the processor is to execute the guest code image within the first key domain using second guest processor state information from the modified guest control structure.

11. The apparatus of claim 1 wherein

the untrusted virtual machine monitor is to issue a load command to the processor to load the encrypted guest control structure into the memory, the load command comprising a pointer to a physical address in the memory from which to load the encrypted guest control structure and a key domain identifier for the first key domain;

and

in response to receiving the load command, the processor is to determine a corresponding key domain key for the key domain identifier, wherein the processor is to use the corresponding key domain key to decrypt the encrypted guest control structure.

12. The apparatus of claim 1, wherein

the processor is further to confirm integrity of the encrypted guest control structure.

13. A processor to:

execute an untrusted host virtual machine monitor to manage execution by the processor of at least one guest virtual machine;

create a first key domain in response to a create command issued by the untrusted host virtual machine monitor, the first key domain comprising a region of a memory to be encrypted by a key domain key, the key domain key inaccessible to the untrusted host virtual machine monitor;

decrypt an encrypted key domain key received from the untrusted host virtual machine monitor to produce the key domain key;

launch a first guest virtual machine within the first key domain in response to a launch command issued by the untrusted host virtual machine monitor, wherein to launch the first guest virtual machine comprises to;

switch to the first key domain,

decrypt an encrypted guest control structure received from the untrusted host virtual machine monitor to produce a guest control structure containing guest processor state information, wherein the guest control structure specifies a protected location of the memory where the processor may store the guest processor state information,

decrypt an encrypted guest code image received from the untrusted host virtual machine monitor to produce a guest code image, and

execute the guest code image within the first key domain using the guest processor state information; and

wherein the processor is further to;

save the guest processor state information for the first guest virtual machine in the protected location of the memory in response to an event triggering an exit condition of the first guest virtual machine; and

in response to receiving a resume command from the untrusted host virtual machine monitor, the processor is to (a) switch to the first key domain, (b) retrieve the guest processor state information for the first guest virtual machine from the protected location of the memory, and (c) execute the guest code image within the first key domain using the guest processor state information.

14. The processor of claim 13, wherein the processor is further to:

switch from the first key domain to a second key domain in response to the event triggering the exit condition of the first guest virtual machine.

15. The processor of claim 13, wherein the processor is further to:

convert the exit condition of the first virtual machine to an exception.

16. The processor of claim 13, wherein the processor is further to:

decrypt an encrypted updated guest control structure to produce an updated guest control structure; and

enter the first guest virtual machine using the updated guest control structure in response to receiving an enter command to enter the first guest virtual machine.

17. The processor of claim 16, wherein the processor is further to:

decrypt an encrypted updated guest code image update to produce an updated guest code image; and

enter the first guest virtual machine by executing the updated guest code image using the updated guest control structure.

18. The processor of claim 13, wherein the processor is further to:

in response to receiving a second launch command to launch a second guest virtual machine within the first key domain, the second guest virtual machine to provide an agent to act on behalf of the untrusted host virtual machine monitor within the first key domain, the processor is to;

switch to the first key domain,

decrypt the encrypted guest code image to produce an agent code image,

decrypt the encrypted guest control structure to produce an agent control structure containing agent processor state information, and

execute the agent code image within the first key domain using the agent processor state information.

19. The processor of claim 13, wherein the processor is further to:

determine a corresponding key domain key for a key domain identifier for the first key domain, wherein the processor is to use the corresponding key domain key to decrypt the encrypted guest control structure in response to receiving a load command to load the encrypted guest control structure into the memory, the load command comprising a pointer to a physical address in the memory from which to load the encrypted guest control structure and the key domain identifier for the first key domain.

20. At least one non-transitory computer-readable medium comprising instructions that, when executed by a processor of a computer, cause a computer to:

receive an encrypted key domain key, an encrypted guest code image encrypted by a key domain key, and an encrypted guest control structure encrypted by the key domain key, wherein the encrypted key domain key comprises an encrypted version of the key domain key;

issue a create command to the processor to create a first key domain, the first key domain comprising a region of a memory to be encrypted by the key domain key;

issue a launch command to the processor to launch a first guest virtual machine within the first key domain, wherein to launch the first guest virtual machine comprises to;

switch to the first key domain,

decrypt the encrypted guest control structure to produce a guest control structure containing guest processor state information, wherein the guest control structure specifies a protected location of the memory where the processor may store the guest processor state information,

decrypt the encrypted guest code image to produce a guest code image, and

execute the guest code image within the first key domain using the guest processor state information;

save the guest processor state information for the first guest virtual machine in the protected location of the memory in response to an event triggering an exit condition of the first guest virtual machine; and

issue a resume command to the processor to resume the first guest virtual machine, wherein to resume the first guest virtual machine comprises to;

switch to the first key domain,

retrieve the guest processor state information for the first guest virtual machine from the protected location of the memory, and

execute the guest code image within the first key domain using the guest processor state information.

21. The at least one computer-readable medium of claim 20, wherein the instructions further cause the computer to:

verify the encrypted guest control structure.

22. The at least one computer-readable medium of claim 20, wherein the instructions further cause the computer to:

intercept an interrupt;

save processor register information to the protected location of the memory in response to at least one of the interrupt and an exception thrown when the first guest virtual machine causes an exit condition;

clear a first processor register if the first processor register is not needed by an untrusted host virtual machine monitor managing execution of the first guest virtual machine;

conditionally expose a second processor register if the second processor register is needed by the untrusted host virtual machine monitor;

invoke the untrusted host virtual machine monitor; and

exit the first guest virtual machine when the untrusted host virtual machine monitor is invoked.

23. The at least one computer-readable medium of claim 20, wherein the instructions further cause the computer to:

receive an encrypted updated guest control structure, install the encrypted updated guest control structure in the memory, and verify the encrypted updated guest control structure; and

issue an enter command to the processor to enter the first guest virtual machine using an updated guest control structure in response to verifying the encrypted updated guest control structure, the updated guest control structure produced by the processor decrypting the encrypted updated guest control structure.

24. The at least one computer-readable medium of claim 20, wherein the instructions further cause the computer to:

determine whether a change to the guest control structure is needed;

verify, by the first virtual machine, that the change to the guest control structure does not compromise security of the first guest virtual machine;

produce, by the first virtual machine, an encrypted updated guest control structure incorporating the change using the key domain key; and

send, by the first virtual machine, the encrypted updated guest control structure to an untrusted host virtual machine monitor via a shared region of memory shared by the untrusted host virtual machine monitor and the first guest virtual machine.

25. The at least one computer-readable medium of claim 24, wherein the instructions further cause the computer to:

install the encrypted updated guest control structure in the memory;

verify the encrypted updated guest control structure; and

issue an enter command to the processor to enter the first guest virtual machine using an updated guest control structure in response to verifying the encrypted updated guest control structure, the updated guest control structure produced by the processor decrypting the encrypted updated guest control structure.

26. The at least one computer-readable medium of claim 20, wherein the instructions further cause the computer to:

verify an agent control structure included within the encrypted guest control structure; and

issue a second launch command to the processor to launch a second guest virtual machine within the first key domain using the agent control structure, the second guest virtual machine to provide an agent to act on behalf of an untrusted host virtual machine monitor within the first key domain.

27. The at least one computer-readable medium of claim 26, wherein the instructions further cause the computer to:

communicate a request to modify the guest control structure of the first guest virtual machine to the agent via a shared region of memory shared with the agent;

modify, by the agent, the guest control structure of the first guest virtual machine within the first key domain to produce a modified guest control structure of the first guest virtual machine in response to reading the request from the shared region of memory;

verify the modified guest control structure of the first guest virtual machine; and

issue an enter command to the processor to enter the first guest virtual machine within the first key domain using the modified guest control structure upon verifying the modified guest control structure.

28. A method comprising:

receiving an encrypted key domain key, an encrypted guest code image encrypted by a key domain key, and an encrypted guest control structure encrypted by the key domain key, wherein the encrypted key domain key comprises an encrypted version of the key domain key;

issuing a create command to a processor create a first key domain, the first key domain comprising a region of a memory to be encrypted by the key domain key;

issuing a launch command to the processor to launch a first guest virtual machine within the first key domain, wherein to launch the first guest virtual machine comprises to;

switch to the first key domain,

decrypt the encrypted guest control structure to produce a guest control structure containing guest processor state information, wherein the guest control structure specifies a protected location of the memory where the processor may store the guest processor state information,

decrypt the encrypted guest code image to produce a guest code image, and

execute the guest code image within the first key domain using the guest processor state information;

saving the guest processor state information for the first guest virtual machine in the protected location of the memory in response to an event triggering an exit condition of the first guest virtual machine; and

issuing a resume command to the processor to resume the first guest virtual machine, wherein to resume the first guest virtual machine comprises to;

switch to the first key domain,

retrieve the guest processor state information for the first guest virtual machine from the protected location of the memory, and

execute the guest code image within the first key domain using the guest processor state information.

29. The method of claim 28, further comprising:

intercepting an interrupt;

saving processor register information to the protected location of the memory in response to at least one of the interrupt and an exception thrown when the first guest virtual machine causes an exit condition;

clearing a first processor register if the first processor register is not needed by an untrusted host virtual machine monitor managing execution of the first guest virtual machine;

conditionally exposing a second processor register if the second processor register is needed by the untrusted host virtual machine monitor;

invoking the untrusted host virtual machine monitor; and

exiting the first guest virtual machine when the untrusted host virtual machine monitor is invoked.

30. The method of claim 28, further comprising:

receiving an encrypted updated guest control structure, installing the encrypted updated guest control structure in the memory, and verifying the encrypted updated guest control structure; and

issuing an enter command to the processor to enter the first guest virtual machine using an updated guest control structure in response to verifying the encrypted updated guest control structure, the updated guest control structure produced by the processor decrypting the encrypted updated guest control structure.

31. The method of claim 30, further comprising:

receiving an encrypted updated guest code image; and

installing the encrypted updated guest code image in the memory.

32. The method of claim 28, further comprising:

determining whether a change to the guest control structure is needed;

verifying, by the first virtual machine, that the change to the guest control structure does not compromise security of the first guest virtual machine;

producing, by the first virtual machine, an encrypted updated guest control structure incorporating the change using the key domain key; and

sending, by the first virtual machine, the encrypted updated guest control structure to an untrusted host virtual machine monitor via a shared region of memory shared by the untrusted host virtual machine monitor and the first guest virtual machine.

33. The method of claim 32, further comprising:

installing the encrypted updated guest control structure in the memory;

verifying the encrypted updated guest control structure; and

issuing an enter command to the processor to enter the first guest virtual machine using an updated guest control structure in response to verifying the encrypted updated guest control structure, the updated guest control structure produced by the processor decrypting the encrypted updated guest control structure.

34. The method of claim 28, further comprising:

verifying an agent control structure included within the encrypted guest control structure; and

issuing a second launch command to the processor to launch a second guest virtual machine within the first key domain using the agent control structure, the second guest virtual machine to provide an agent to act on behalf of an untrusted host virtual machine monitor within the first key domain.

35. The method of claim 34, further comprising:

communicating a request to modify the guest control structure of the first guest virtual machine to the agent via a shared region of memory shared with the agent;

modifying, by the agent, the guest control structure of the first guest virtual machine within the first key domain to produce a modified guest control structure of the first guest virtual machine in response to reading the request from the shared region of memory;

verifying the modified guest control structure of the first guest virtual machine; and

issuing an enter command to the processor to enter the first guest virtual machine within the first key domain using the modified guest control structure upon verifying the modified guest control structure.