Systems and methods for supporting hot plugging of remote storage devices accessed over a network via NVME controller

• US 9,819,739 B2
• Filed: 09/25/2014
• Issued: 11/14/2017
• Est. Priority Date: 05/02/2014
• Status: Active Grant

• Current Assignee: Marvell Asia Pte Limited (Marvell Technology Group Limited)
• Original Assignee: Cavium, LLC (Marvell Technology Group Limited)
• Inventors: Hussain, Muhammad Raghib, Murgai, Vishal, Panicker, Manojkumar, Masood, Faisal, Folsom, Brian, Kessler, Richard Eugene
• Primary Examiner(s): Avellino, Joseph E
• Assistant Examiner(s): MCLEOD, MARSHALL M
• Application Number:
  US14/496,916
• Publication Number:
  US 20150319243A1
• Time in Patent Office: 1,146 Days
• Field of Search: 709201, 709204, 709216, 709217
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45587 Isolation or security of virtual machine instances
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0662 Virtualisation aspects
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/0866 Checking the configuration
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A new approach is proposed that contemplates systems and methods to support hot plugging and/or unplugging one or more of remote storage devices virtualized as extensible/flexible storages and NVMe namespace(s) via an NVMe controller during operation. First, the NVMe controller virtualizes and presents a set of remote storage devices to one or more VMs running on a host attached to the NVMe controller as logical volumes in the NVMe namespace(s) so that each of the VMs running on the host can access these remote storage devices to perform read/write operations as if they were local storage devices. When the one or more remote storage devices are added to or removed from the set of remote storage devices based on storage space needs of the VMs, the NVMe controller updates the logical volumes in the NVMe namespace(s) accordingly and enables these remote storage devices to be hot plugged or unplugged from the plurality of remote storage devices at runtime without requiring shutting down and restarting any of the VMs, the host, and/or the NVMe controller. The VMs may then perform read/write operations on the NVMe namespace(s) updated to reflect the changes in the configuration of the set of remote storage devices dynamically without any interruption.

Claims

1. A system to support hot plugging of virtualized remote storage devices, comprising:

a physical NVMe controller having one or more processors, which in operation, is configured to;

create and map one or more logical volumes in one or more local NVMe namespaces to a set of remote storage devices over a network via the NVMe controller following a storage network protocol, where the logical volumes mapped to the remote storage devices are accessible by one or more virtual machines (VMs) running on a host over the network via the NVMe controller as if they were local storage volumes coupled to the NVMe controller;

monitor configuration of the set of remote storage devices accessible over the network for any addition to or removal from the set of remote storage devices;

update the logical volumes in the NVMe namespaces mapped to the set of remote storage devices over the network accordingly when one or more remote storage devices are added to or removed from the set of remote storage devices at runtime;

present the logical volumes in the NVMe namespaces updated to reflect the added or removed remote storage devices to the VMs running on the host;

enable the VMs running on the host to perform one or more read and/or write operations on the updated logical volumes without the host and/or the NVMe controller needing to be shut down or restarted.

2. The system of claim 1, wherein:

the host of the VMs is an x86 or ARM server.

3. The system of claim 1, wherein:

the logical volumes further include storage devices attached to the physical NVMe controller locally.

4. The system of claim 1, wherein:

the VMs run independently on the host and are isolated from each other so that one VM cannot access the data and/or communication of any other VMs running on the same host.

5. The system of claim 1, wherein:

the storage network protocol is Internet Small Computer System Interface (iSCSI).

6. The system of claim 1, wherein:

multiple of the plurality of VMs are enabled to access the same logical volume and each logical volume is enabled to be shared among the multiple VMs.

7. The system of claim 1, wherein:

the NVMe controller is configured to perform the mapping on the remote storage devices over the network transparent to the VMs.

8. The system of claim 1, wherein:

the NVMe controller is configured to monitor and collect volume information of the added or moved remote storage devices over the network.

9. The system of claim 1, wherein:

the NVMe controller is configured to monitor access to the logical volumes and namespaces mapped to the remote storage devices that are in active use by the VMs running on the host to prioritize data traffic to/from those remote storage devices so that the VMs have ready access to these logical volumes and namespaces.

10. The system of claim 9, wherein:

the NVMe controller is configured to pre-fetch data from a volume of the remote storage devices that are most frequently accessed by the VMs to a cache locally associated with the NVMe controller in anticipation of the next read operation by the VMs.

11. The system of claim 1, wherein:

the NVMe controller is configured to save context of the logical volume and the namespace previously used by one of the VMs for future use and make a new logical volume and namespace available to the VM when the VM switches to use a different logical volume or namespace.

12. The system of claim 1, wherein:

the NVMe controller is configured to establish a lookup table that maps between the NVMe namespaces of the logical volumes and the remote physical storage volumes that are imported to the NVMe controller.

13. The system of claim 12, wherein:

the NVMe controller is configured to maintain an inactivity timer/counter on the logical volumes and namespaces mapped to the remote storage devices and to purge one or more logical volumes and namespaces that have not been accessed by the VMs for a certain period of time from the lookup table and the logical volumes and namespaces to be presented to the VMs.

14. The system of claim 13, wherein:

the remote storage devices mapped to the logical volumes and namespaces being purged are removed and/or hot-unplugged from the set of the remote storage devices.

15. The system of claim 1, wherein:

the NVMe controller is configured to notify the VMs accessing logical volumes in the namespaces mapped to one or more of the remote storage devices that one or more of the remote storage devices are unreachable over the network and to update the logical volumes in the namespaces to remove the volumes mapped to the unreachable remote storage devices.

16. A computer-implemented method to support hot plugging of virtualized remote storage devices, comprising:

creating and mapping one or more logical volumes in one or more local NVMe namespaces to a set of remote storage devices over a network via an NVMe controller, where the logical volumes mapped to the remote storage devices are accessible by one or more virtual machines (VMs) running on a host over the network via the NVMe controller as if they were local storage volumes coupled to the NVMe controller;

monitoring configuration of the set of remote storage devices accessible over the network for any addition to or removal from the set of remote storage devices;

updating the logical volumes in the NVMe namespaces mapped to the set of remote storage devices over the network accordingly when one or more remote storage devices are added to or removed from the set of remote storage devices at runtime;

presenting the logical volumes in the NVMe namespaces updated to reflect the added or removed remote storage devices to the VMs running on the host;

enabling the VMs running on the host to perform one or more read and/or write operations on the updated logical volumes without the host and/or the NVMe controller needing to be shut down or restarted.

17. The method of claim 16, further comprising:

enabling the VMs to run independently on the host and isolating the VMs from each other so that one VM cannot access the data and/or communication of any other VMs running on the same host.

18. The method of claim 16, further comprising:

enabling multiple of the plurality of VMs to access the same logical volume and each logical volume to be shared among the multiple VMs.

19. The method of claim 16, further comprising:

performing the mapping on the remote storage devices over the network transparent to the VMs.

20. The method of claim 16, further comprising:

monitoring and collecting volume information of the added or moved remote storage devices over the network.

21. The method of claim 16, further comprising:

monitoring access to the logical volumes and namespaces mapped to the remote storage devices that are in active use by the VMs running on the host to prioritize data traffic to/from those remote storage devices so that the VMs have ready access to these logical volumes and namespaces.

22. The method of claim 21, further comprising:

pre-fetching data from a volume of the remote storage devices that are most frequently accessed by the VMs to a cache locally associated with the NVMe controller in anticipation of the next read operation by the VMs.

23. The method of claim 16, further comprising:

saving context of the logical volume and the namespace previously used by one of the VMs for future use and make a new logical volume and namespace available to the VM when the VM switches to use a different logical volume or namespace.

24. The method of claim 16, further comprising:

establishing a lookup table that maps between the NVMe namespaces of the logical volumes and the remote physical storage volumes.

25. The method of claim 24, further comprising:

maintaining an inactivity timer/counter on the logical volumes and namespaces mapped to the remote storage devices and to purge one or more logical volumes and namespaces that have not been accessed by the VMs for a certain period of time from the lookup table and the logical volumes and namespaces to be presented to the VMs.

26. The method of claim 25, further comprising:

removing and/or hot-unplugging the remote storage devices mapped to the logical volumes and namespaces being purged from the set of the remote storage devices.

27. The method of claim 16, further comprising:

notifying the VMs accessing logical volumes in the namespaces mapped to one or more of the remote storage devices that one or more of the remote storage devices are unreachable over the network and to update the logical volumes in the namespaces to remove the volumes mapped to the unreachable remote storage devices.