Architecture for managing I/O and storage for a virtualization environment

• US 8,863,124 B1
• Filed: 08/09/2012
• Issued: 10/14/2014
• Est. Priority Date: 08/10/2011
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Aron, Mohit
• Primary Examiner(s): SWIFT, CHARLES M
• Application Number:
  US13/571,188
• Time in Patent Office: 796 Days
• Field of Search: None
• US Class Current: 718/1
• CPC Class Codes: G06F 13/4221 being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 3/0662 Virtualisation aspects
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 45/586 of virtual routers

Abstract

Disclosed is an improved approach to implement I/O and storage device management in a virtualization environment. According to some approaches, a Service VM is employed to control and manage any type of storage device, including directly attached storage in addition to networked and cloud storage. The Service VM implements the Storage Controller logic in the user space, and can be migrated as needed from one node to another. IP-based requests are used to send I/O request to the Service VMs. The Service VM can directly implement storage and I/O optimizations within the direct data access path, without the need for add-on products.

Claims

1. A system for communicating to a storage controller in a virtualization environment, comprising:

a plurality of nodes, wherein the plurality of nodes implements a virtualization environment, and comprises a hypervisor and multiple user virtual machines;

a plurality of hardware storage devices that are accessed by the multiple user virtual machines; and

a storage controller implemented as a service virtual machine on each node of the plurality of nodes to manage access by the user virtual machines to the plurality of storage devices;

wherein each service virtual machine is dedicated to its corresponding node;

wherein a request for storage managed by a service virtual machine at another node is handled by the service virtual machine at the other node;

wherein for a node of the plurality of nodes, a first virtual switch is employed to handle internal network traffic with the service virtual machine on the node and a second virtual switch is employed to handle external network traffic with the service virtual machine on the node.

2. The system of claim 1, wherein an IP-based protocol is utilized to communicate the internal network traffic with the service virtual machine.

3. The system of claim 1, wherein the IP-based protocol is an Internet Small Computer System Interface (iSCSI) protocol.

4. The system of claim 1, wherein the IP-based protocol is a network file system (NFS) protocol.

5. The system of claim 1, wherein the service virtual machine on the node comprises:

a first virtual network interface card (NIC) for routing the internal network traffic; and

a second virtual network interface card (NIC) for routing the external network traffic.

6. The system of claim 1, wherein the service virtual machine on the node runs as a virtual machine above the hypervisor.

7. The system of claim 1, in which the service virtual machine on the node manages a virtual disk that is exposed to the multiple user virtual machines.

8. The system of claim 1 in which the service virtual machine for each of the plurality of nodes corresponds to a same IP address isolated by an internal VLAN.

9. The system of claim 1, in which a new node that is added to the system corresponds to a new service virtual machine that acts as the storage controller for the new node.

10. The system of claim 8, in which the virtual disk corresponds to one or more block devices or server targets.

11. The system of claim 1 in which the service virtual machine on the node comprises an I/O director module, wherein the I/O director module operates to intercept a request from a user virtual machine and to format the request into a virtual disk request.

12. The system of claim 11 in which the I/O director module comprises an admission control module that determines whether the service virtual machine is permitted to operate upon data storage identified in a request.

13. The system of claim 11 in which the I/O director module comprises a virtual disk controller that implements read and write requests.

14. The system of claim 11 in which the I/O director module performs quality of service (QoS) optimizations for a service request.

15. The system of claim 14 in which the QoS optimizations are implemented by establishing multiple request queues for different priority levels of requests.

16. The system of claim 1 in which the service virtual machine on the node comprises a distributed metadata service module to maintain metadata for virtual disks managed by the service virtual machine on the node.

17. The system of claim 1 in which the service virtual machine on the node comprises a health management module to maintain consistency of metadata for virtual disks managed by the service virtual machine on the node.

18. The system of claim 1 in which the service virtual machine on the node comprises a distributed configuration database module to maintain configuration data for the service virtual machine on the node.

19. The system of claim 18 in which the distributed configuration database module operates to maintain health information for entities in a system, to handle elections, or to perform consensus management.