MemX: virtualization of cluster-wide memory

• US 8,935,506 B1
• Filed: 03/22/2012
• Issued: 01/13/2015
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: The Research Foundation for The State University of New York (State University of New York)
• Original Assignee: The Research Foundation for The State University of New York (State University of New York)
• Inventors: Gopalan, Kartik
• Primary Examiner(s): Li, Aimee
• Assistant Examiner(s): DO, CHRISTOPHER N
• Application Number:
  US13/427,110
• Time in Patent Office: 1,027 Days
• Field of Search: 711/202
• US Class Current: 711/202
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures

Abstract

MemX provides a distributed system that virtualizes cluster-wide memory to support data-intensive and large memory workloads in virtual machines (VMs), and provides benefits in virtualized settings: (1) VM workloads that access large datasets can perform low-latency I/O over virtualized cluster-wide memory; (2) VMs can transparently execute very large memory applications that require more memory than physical DRAM present in the host machine; (3) reduces the effective memory usage of the cluster by de-duplicating pages that have identical content; (4) existing applications do not require any modifications to benefit from MemX such as the use of special APIs, libraries, recompilation, or relinking; and (5) supports live migration of large-footprint VMs by eliminating the need to migrate part of their memory footprint resident on other nodes. Detailed evaluations show that large dataset applications and multiple concurrent VMs achieve significant performance improvements using MemX compared against virtualized local and iSCSI disks.

Claims

1. A distributed virtual memory system, comprising:

a block device interface, providing an interface between a first memory system of a client and a virtual distributed memory system having a linear I/O space;

a module configured to map the linear I/O space of the block device interface to a second memory system of at least one remote server, wherein memory pages are defined for communication through a network interface between the client and the at least one remote server, wherein a plurality of pages comprise a page group, and the module is further configured to inferentially determine a storage location of a memory page based on a logical position of the memory page within the page group and an algorithm; and

a write buffer configured to service write requests.

2. The system according to claim 1, further comprising a hash table configured to determine duplicated memory pages.

3. The system according to claim 2, wherein a duplicated memory page is represented by an alias.

4. The system according to claim 1, further comprising a parity generator, wherein a set comprising a plurality of memory pages are processed to generate at least one parity page.

5. The system according to claim 4, wherein the module is configured to store the plurality of memory pages and the at least one parity page on different remote servers.

6. The system according to claim 1, wherein a memory page comprises metadata, the metadata representing an aliased memory page resulting from deduplication.

7. The system according to claim 4, further comprising a fault recovery module configured to generate a memory page of a respective set based on other memory pages and at least one parity page of the set.

8. The system according to claim 1, wherein the module is configured to store a memory page on a remote server, evict the memory page from a client, retrieve the memory page from the remote server, and restore the memory page on the client.

9. The system according to claim 8, wherein the client comprises a virtual machine, and the module comprises a logical process within the virtual machine.

10. The system according to claim 8, wherein the client comprises a virtual machine, and the module comprises a logical process outside the virtual machine.

11. The system according to claim 10, wherein the module implements memory page deduplication for a plurality of virtual machines concurrently.

12. The system according to claim 1, wherein the at least one remote server comprises:

a resource discovery transmission module, configured to periodically transmit resource announcement messages through a computer network, indicating available server resources and load;

a serving module, configured to generate a communication to the module, comprising a server response to a task to which is appended the indicated available server resources and load; and

a migration module, configured to migrate tasks consuming the server resources to another server module across the computer network, and to communicate an update identifying the other server module to which the tasks are migrated.

13. The server system according to claim 12, wherein the client asks comprise memory page storage and retrieval.

14. The server system according to claim 13, wherein the serving module is configured to provide data deduplication and representation of duplicated data with an index identifying a storage location of the data.

15. The server system according to claim 14, wherein the serving module is further configured to regenerate duplicated data when either the duplicated data is requested or the data is modified.

16. The server system according to claim 12, wherein the serving module predicts a task.

17. The server system according to claim 16, wherein the task comprises a memory access, and wherein the server module is configured to proactively retrieve a predicted required memory page.

18. The server system according to claim 12, further comprising a distributed process coordination module, configured to communicate with other server systems and coordinate server responses to correlated tasks.

19. The server system according to claim 12, wherein the tasks comprise virtual memory storage and retrieval operations emulating a block device interface.

20. The server system according to claim 19, wherein the tasks are generated within a virtual machine environment.

21. The server system according to claim 19, wherein the tasks are generated partially within and partially outside a virtual machine environment.

22. The server system according to claim 12, wherein the server system is configured to process tasks for a plurality of clients.

23. The server system according to claim 22, wherein the tasks comprise memory storage and retrieval operations, further comprising a parity module, and a deduplication module, wherein the server system is configured to determine whether a memory page stored on the server system is duplicated, to generate an alias of a duplicated page, to store as metadata in a retained copy of the duplicated memory page an index of a duplicated page which is not retained, and to update parity information for the retained copy of the duplicated memory page.

24. The server system according to claim 23, wherein the parity information is stored on at least one other server system, and wherein the server system is further configured to retrieve a copy of other memory pages which are represented in joint parity information, to compute new parity information, and to store the new parity information in place of the prior parity information.

25. The server system according to claim 12, wherein the migration module is configured to instantiate a new virtual machine, and then to transfer responsibility for tasks to the new virtual machine, wherein the server system and the new virtual machine concurrently handle respective tasks during a period in which less than all of the client tasks are migrated.

26. A virtual memory system comprising:

a plurality of virtual machines configured to execute on common physical hardware having a memory;

a common device driver domain, shared by the plurality of virtual machines;

a hypervisor, coordinating the execution of the plurality of virtual machines and the common device driver domain;

a network interface; and

a virtual memory module, configured to at least one of;

execute within a respective virtual machine and provide an interface to the common device driver configured as a large virtual memory for the respective virtual machine, supporting communication through the network interface controlled by the common device driver domain;

execute within the common driver domain and service the plurality of virtual machines, providing a block memory device interface for each respective virtual machine; and

execute within the hypervisor, and provide a virtual memory interface for the hypervisor, which is reported to the plurality of virtual machines as a large available memory pool,

wherein the virtual memory module stores and retrieves information as memory pages wherein a plurality of memory pages together comprise a page group, and inferentially determines a storage location of a respective memory page of the page group selectively based on a logical position of the memory page within the page group and an algorithm.

27. A server module for a distributed virtual memory system, comprising:

a resource discovery transmission module, configured to periodically send resource announcement messages through a computer network, providing an indication of available local memory to the server module and load;

a page serving module, configured to retrieve a page from local memory, append to the page an indication of available local memory to the server module and load, and transfer the appended page through the computer network;

a page migration module, configured to migrate a page of memory to another server module across the computer network, and to send a message to a respective client to provide an update of a location of the migrated page; and

wherein a plurality pages comprise a page group, and the page migration module inferentially determines a storage location of a respective page selectively based on a logical position of the respective page within the page group and an algorithm.