System and method for security and privacy aware virtual machine checkpointing

• US 10,324,795 B2
• Filed: 01/24/2017
• Issued: 06/18/2019
• Est. Priority Date: 10/01/2012
• Status: Active Grant

• Current Assignee: The Research Foundation for The State University of New York (State University of New York)
• Original Assignee: Research Foundation For The State University O
• Inventors: Yang, Ping, Gopalan, Kartik
• Primary Examiner(s): Sun, Scott C
• Application Number:
  US15/414,404
• Publication Number:
  US 20170132084A1
• Time in Patent Office: 875 Days
• CPC Class Codes: G06F 11/1407 Checkpointing the instruction stream
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/6245 Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 21/79 in semiconductor storage media, e.g. directly-addressable memories
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A checkpointing method for creating a file representing a restorable state of a virtual machine in a computing system, comprising identifying processes executing within the virtual machine that may store confidential data, and marking memory pages and files that potentially contain data stored by the identified processes; or providing an application programming interface for marking memory regions and files within the virtual machine that contain confidential data stored by processes; and creating a checkpoint file, by capturing memory pages and files representing a current state of the computing system, which excludes information from all of the marked memory pages and files.

Claims

1. A method of performing a checkpointing process within a virtual machine, comprising:

(a) identifying a set of memory pages comprising user space memory pages and kernel space memory pages occupied by at least one application executing under control of a hypervisor of the virtual machine;

(b) distinguishing a first subset of the set of memory pages comprising private user space memory pages and private kernel space memory pages occupied by the at least one application which represent private information from a second subset of the set of memory pages comprising the user space memory pages, and the kernel space memory pages occupied by the at least one application which do not represent the private information to the hypervisor; and

(c) persistently storing a checkpoint file representing a state of the hypervisor comprising the identifications of the second subset of the set of memory pages, substantially without the first subset of the set of memory pages.

2. The method according to claim 1, further comprising restoring a prior state of the hypervisor from the persistently stored checkpoint file, from the first second subset of memory pages and without the first subset of memory pages, and resuming the at least one application which are not dependent on the first subset of the set of memory pages.

3. The method according to claim 1, further comprising freezing all processes of the at least one application in a user space, except the checkpointing process, while the checkpoint file is created.

4. The method according to claim 1, further comprising determining inter-process dependencies comprising information in TTY buffers, socket buffers, pipe buffers, and FIFO buffers, wherein the checkpoint file further comprises information for resuming the at least one application which, based on the inter-process dependencies, comprise processes which are not dependent on the first subset of the set of memory pages.

5. The method according to claim 1, further comprising segregating a first process container which encapsulates at least all memory pages associated with a respective application, separate from a second process container which encapsulates memory pages associated with another respective application, wherein the first process container and the second process container are separately stored within the checkpoint file.

6. The method according to claim 1, further comprising identifying the inter-process dependencies for the at least one application by at least a static analysis.

7. The method according to claim 1, further comprising identifying inter-process dependencies for a plurality of different applications by at least a dynamic analysis.

8. The method according to claim 1, further comprising determining an open communication of the at least one application, and deferring creation of the checkpoint file with respect to the at least one application until completion of the open communication.

9. The method according to claim 1, further comprising:

communicating a message to the at least one application prior to creating the checkpoint file;

assuming a safe state by the at least one application in response to the message, in which private information is removed from at least a portion of the user space memory pages and the kernel space memory pages so that the at least a portion of the user space memory pages and the kernel space memory pages are within the second subset of the set of memory pages;

and

after storing the checkpoint file, reverting the at least one application to a normal operation state.

10. The method according to claim 9, further comprising:

restoring the state of the hypervisor from the checkpoint file; and

informing the at least one application that the hypervisor is restored from the checkpoint file.

11. The method according to claim 1, further comprising, under control of the checkpointing process:

sanitizing kernel space memory by pausing system calls and I/O requests from the at least one application by the kernel;

completing or flushing any pending I/O operations of the at least one application;

removing data from all I/O buffers after the completion of the I/O operations; and

preparing the checkpoint file for storage after said sanitizing, completing or flushing, and removing.

12. A method for checkpointing a virtual machine, comprising performing, by a checkpointing process executing on the virtual machine under control of a hypervisor:

(a) pausing system calls and I/O requests from at least one application, complete or flush pending I/O operations, and zeroing out all I/O buffers after the completion of the I/O operations;

(b) identifying user space memory pages and kernel space memory locations of the at least one application, which contain private information and user space memory pages and kernel space memory locations of the at least one application that exclude private information;

(c) creating a checkpoint file comprising at least the user space memory pages and kernel space memory locations of the at least one application that exclude the private information, and which does not comprise the user space memory pages and kernel space memory locations which contain private information; and

(d) recommencing system calls and I/O requests from the at least one application.

13. The method according to claim 12, further comprising determining inter-process dependencies, wherein the checkpoint file further comprises information for selectively, based on the inter-process dependencies, resuming respective applications upon restoration of the virtual machine from the checkpoint file, which are not dependent on the user space memory pages and kernel space memory locations which contain private information.

14. The method according to claim 13, further comprising determining inter-process dependencies for the at least one application by static analysis and dynamic analysis.

15. The method according to claim 12, further comprising restoring the virtual machine to substantially resume operation, based on the created checkpoint file.

16. The method according to claim 12, further comprising:

monitoring an input-output data stream communication from non-program memory sources to the virtual machine;

storing a series of states representing activities of the at least one application with respect to the monitored input-output data stream communications;

tagging instances of data within the created checkpoint file corresponding to the monitored input-output data stream communication; and

restoring the virtual machine to substantially resume operation, based on the created checkpoint file, wherein a state of the at least one process is rolled back to a state prior to a respective monitored input-output data stream communication.

17. The method according to claim 16, wherein memory pages within the created checkpoint file which contain the tagged instances of data are excluded or obscured.

18. The method according to claim 12, further comprising determining an ongoing communication a respective application, and deferring creation of the checkpoint file with respect to the respective application until completion of the communication.

19. The method according to claim 12, further comprising:

communicating a message to the at least one application prior to creating the checkpoint file;

assuming a safe state by the at least one application, in which confidential information is removed from the memory pages which are captured within the checkpoint file;

sanitizing kernel space memory by pausing system calls and I/O requests from the at least one application by the kernel;

completing or flushing any pending I/O operations of the at least one application;

removing data from all I/O buffers after the completion of the I/O operations; and

after creating the checkpoint file, reverting the at least one application to a normal operation state.

20. A system for performing a checkpointing process within a virtual machine, comprising:

a memory configured to store memory pages of a virtual machine controlled having a hypervisor controlling execution of at least one application;

the hypervisor controlling execution of a set of checkpointing processes, comprising;

(a) a process to identify a set of memory pages comprising user space memory pages and kernel space memory pages occupied by at least one application executing under control of a hypervisor of the virtual machine;

(b) a process to distinguish a first subset of the set of memory pages comprising private user space memory pages and private kernel space memory pages occupied by the at least one application which represent private information from a second subset of the set of memory pages comprising the user space memory pages, and the kernel space memory pages occupied by the at least one application which do not represent the private information to the hypervisor;

(c) a process to persistently store a checkpoint file representing a state of the hypervisor comprising the identifications of the second subset of the set of memory pages, substantially without the first subset of the set of memory pages.