Architecture for managing I/O and storage for a virtualization environment

• US 9,256,456 B1
• Filed: 10/14/2014
• Issued: 02/09/2016
• Est. Priority Date: 08/10/2011
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Aron, Mohit
• Primary Examiner(s): SWIFT, CHARLES M
• Application Number:
  US14/514,326
• Time in Patent Office: 483 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 13/4221 being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 3/0662 Virtualisation aspects
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 45/586 of virtual routers

Abstract

Disclosed is an improved approach to implement I/O and storage device management in a virtualization environment. According to some approaches, a Service VM is employed to control and manage any type of storage device, including directly attached storage in addition to networked and cloud storage. The Service VM implements the Storage Controller logic in the user space, and can be migrated as needed from one node to another. IP-based requests are used to send I/O request to the Service VMs. The Service VM can directly implement storage and I/O optimizations within the direct data access path, without the need for add-on products.

Claims

1. A system for communicating to a storage controller in a virtualization environment, comprising:

a plurality of nodes, wherein the plurality of nodes implements a virtualization environment, and comprises a hypervisor and multiple user virtual machines;

a plurality of hardware storage devices that are accessed by the multiple user virtual machines; and

a first storage controller implemented as a service virtual machine on a first node of the plurality of nodes and a second storage controller implemented as a service virtual machine on a second node of the plurality of nodes to manage access by the user virtual machines to the plurality of storage devices, wherein the service virtual machines are dedicated to their corresponding nodes; and

wherein the first storage controller is configured to identify the second storage controller, and wherein the first storage controller is further configured to create an I/O request to be sent to the second storage controller that was identified by the first storage controller to cooperatively perform storage controller operations for the first node and the second node.

2. The system of claim 1, wherein the first storage controller and the second storage controller utilize an IP-based protocol to cooperatively perform storage controller operations.

3. The system of claim 2, wherein the IP-based protocol is an Internet Small Computer System Interface (iSCSI) protocol.

4. The system of claim 2, wherein the IP-based protocol is a network file system (NFS) protocol.

5. The system of claim 1, wherein the service virtual machine comprises:

a first virtual network interface card (NIC) for routing internal network traffic; and

a second virtual network interface card (NIC) for routing external network traffic.

6. The system of claim 1, wherein the service virtual machines run as a virtual machines above their respective hypervisors.

7. The system of claim 1, in which the service virtual machine manages a virtual disk that is exposed to the multiple user virtual machines.

8. The system of claim 7, in which the virtual disk corresponds to one or more block devices or server targets.

9. The system of claim 1, in which a new node that is added to the system corresponds to a new service virtual machine that acts as the storage controller for the new node.

10. The system of claim 1, in which the first storage controller and the second storage controller correspond to a same IP address isolated by an internal VLAN.

11. A method for communicating to a storage controller in a virtualization environment, comprising:

identifying a plurality of nodes, wherein the plurality of nodes implements a virtualization environment, and comprises a hypervisor and multiple user virtual machines;

implementing a first storage controller as a service virtual machine on a first node of the plurality of nodes and implementing a second storage controller as a service virtual machine on a second node of the plurality of nodes to manage access by the user virtual machines to a plurality of storage devices, wherein the plurality of storage devices are accessed by the multiple user virtual machines, and wherein the service virtual machines are dedicated to their corresponding nodes;

identifying, via the first storage controller, the second storage controller; and

creating, via the first storage controller, an I/O request to be sent to the second storage controller that was identified by the first storage controller to cooperatively perform storage controller operations for the first node and the second node.

12. The method of claim 11, wherein the first storage controller and the second storage controller utilize an IP-based protocol to cooperatively perform storage controller operations.

13. The method of claim 12, wherein the IP-based protocol is an Internet Small Computer System Interface (iSCSI) protocol.

14. The method of claim 12, wherein the IP-based protocol is a network file system (NFS) protocol.

15. The method of claim 11, wherein the service virtual machine comprises:

a first virtual network interface card (NIC) for routing internal network traffic; and

a second virtual network interface card (NIC) for routing external network traffic.

16. The method of claim 11, wherein the service virtual machines run as a virtual machines above their respective hypervisors.

17. The method of claim 11, in which the service virtual machine manages a virtual disk that is exposed to the multiple user virtual machines.

18. The method of claim 17, in which the virtual disk corresponds to one or more block devices or server targets.

19. The method of claim 11, in which a new node that is added to the virtualization environment corresponds to a new service virtual machine that acts as the storage controller for the new node.

20. The method of claim 11 in which the first storage controller and the second storage controller correspond to a same IP address isolated by an internal VLAN.

21. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for communicating to a storage controller in a virtualization environment, comprising:

identifying a plurality of nodes, wherein the plurality of nodes implements a virtualization environment, and comprises a hypervisor and multiple user virtual machines;

implementing a first storage controller as a service virtual machine on a first node of the plurality of nodes and implementing a second storage controller as a service virtual machine on a second node of the plurality of nodes to manage access by the user virtual machines to a plurality of storage devices, wherein the plurality of storage devices are accessed by the multiple user virtual machines, and wherein the service virtual machines are dedicated to their corresponding nodes;

identifying, via the first storage controller, the second storage controller; and

creating, via the first storage controller, an I/O request to be sent to the second storage controller that was identified by the first storage controller to cooperatively perform storage controller operations for the first node and the second node.

22. The computer program product of claim 21, wherein the first storage controller and the second storage controller utilize an IP-based protocol to cooperatively perform storage controller operations.

23. The computer program product of claim 22, wherein the IP-based protocol is an Internet Small Computer System Interface (iSCSI) protocol.

24. The computer program product of claim 22, wherein the IP-based protocol is a network file system (NFS) protocol.

25. The computer program product of claim 21, wherein the service virtual machine comprises:

a first virtual network interface card (NIC) for routing internal network traffic; and

a second virtual network interface card (NIC) for routing external network traffic.

26. The computer program product of claim 21, wherein the service virtual machines run as a virtual machines above their respective hypervisors.

27. The computer program product of claim 21, in which the service virtual machine manages a virtual disk that is exposed to the multiple user virtual machines.

28. The computer program product of claim 27, in which the virtual disk corresponds to one or more block devices or server targets.

29. The computer program product of claim 21, in which a new node that is added to the virtualization environment corresponds to a new service virtual machine that acts as the storage controller for the new node.

30. The computer program product of claim 21, in which the first storage controller and the second storage controller correspond to a same IP address isolated by an internal VLAN.