Storage area network emulation

• US 9,798,497 B1
• Filed: 06/08/2016
• Issued: 10/24/2017
• Est. Priority Date: 06/08/2015
• Status: Active Grant

• Current Assignee: Skytap Incorporated
• Original Assignee: Skytap Incorporated
• Inventors: Schick, Bradley M., Shelepov, Bulat, Slioussar, Nikolai
• Primary Examiner(s): Padmanabhan, Mano
• Assistant Examiner(s): Edouard, Jean
• Application Number:
  US15/177,266
• Time in Patent Office: 503 Days
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 9/4406 Loading of operating system
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Generating a virtual storage area network (VSAN) is disclosed. From the perspective of a customer'"'"'s virtual machine, the VSAN is a logical network of storage devices that provide features that are typically associated with a physical storage access network, such as block level data storage; logical disk arrays; tape libraries; optical jukeboxes; quality of service; disk mirroring, backup and restoration services; archival and retrieval of archived data; data migration from one virtual storage device to another; sharing of data among different virtual machines in a network; and the incorporation of virtual subnetworks.

Claims

1. A computer-implemented method, comprising:

for each of a first virtual storage area network and a second virtual storage area network;

receiving metadata defining at least one storage pool configuration and at least one virtual machine configuration;

establishing the virtual storage area network by;

allocating a virtual machine in accordance with the virtual machine configuration defined by the metadata received for the virtual machine;

configuring the allocated virtual machine to represent to other virtual machines as a mass storage device an allocation of data storage at a communicatively-coupled physical block storage device, in accordance with the storage pool configuration specified by the metadata received for the virtual machine; and

booting an operating system of the virtual machine, wherein the operating system is configured based at least on the metadata, wherein the metadata includes an entry for an I/O data rate for sending storage requests from the virtual machine to the storage pool, and wherein a network controller of the virtual machine is configured based on the entry.

2. A computer-implemented method, comprising:

for each of a first virtual storage area network and a second virtual storage area network;

receiving metadata defining at least one storage pool configuration and at least one virtual machine configuration;

establishing the virtual storage area network by;

allocating a virtual machine in accordance with the virtual machine configuration defined by the metadata received for the virtual machine;

configuring the allocated virtual machine to represent to other virtual machines as a mass storage device an allocation of data storage at a communicatively-coupled physical block storage device, in accordance with the storage pool configuration specified by the metadata received for the virtual machine; and

in response to receiving storage access requests at a first I/O data rate, buffering one or more of the storage access requests in a network controller of the virtual machine, wherein at least one of the one or more buffered storage access requests is sent from the virtual machine to the storage pool at a second I/O data rate that is equal to or less than a third I/O data rate defined in the metadata.

3. A computer-implemented method comprising:

for each of a first virtual storage area network and a second virtual storage area network;

receiving metadata defining at least one storage pool configuration and at least one virtual machine configuration;

establishing the virtual storage area network by;

allocating a virtual machine in accordance with the virtual machine configuration defined by the metadata received for the virtual machine;

configuring the allocated virtual machine to represent to other virtual machines as a mass storage device an allocation of data storage at a communicatively-coupled physical block storage device, in accordance with the storage pool configuration specified by the metadata received for the virtual machine; and

detecting a change to the metadata, and in response to the detected change, reconfiguring the virtual storage area network based on the detected change.

4. The computer-implemented method of claim 3, wherein the metadata defines a virtual size of the storage pool and a physical size of the storage pool.

5. The computer-implemented method of claim 3, wherein the metadata defines an I/O data rate limit for sending storage block access requests to the storage pool.

6. The computer-implemented method of claim 3, wherein storage block access requests are received at the virtual machine, wherein the virtual machine verifies an I/O data rate associated with the storage block requests, and wherein the storage block requests are sent from the virtual machine to the storage pool at an I/O data rate defined in the metadata.

7. The computer-implemented method of claim 3, wherein storage access requests are sent from a customer'"'"'s virtual machine for delivery to the storage pool at a first I/O data rate, wherein data in the metadata defines a maximum I/O data rate for sending data to the storage pool, and wherein the first I/O data rate for sending the storage access requests to the storage pool is dynamically modified to an I/O data rate less than or equal to the maximum I/O data rate associated with the storage pool.

8. The computer-implemented method of claim 3, further comprising determining a new virtual storage area network, wherein the new virtual storage area network includes a new virtual machine communicatively coupled to a new storage pool at the physical block storage device.

9. The computer-implemented method of claim 3, further comprising:

booting a commodity operating system of the virtual machine, wherein the commodity operating system is configured based at least on the metadata, and wherein the metadata includes an entry for an I/O data rate for sending storage requests from the virtual machine to the storage pool.

10. The computer-implemented method of claim 3, further comprising:

configuring a network controller of the virtual machine to traffic shape storage requests sent to the one or more storage pools, wherein the traffic shaping is based on the metadata.

11. A virtual storage area network, comprising:

a first virtual machine;

a second virtual machine; and

a storage pool,

wherein the first virtual machine is programmed to;

send data to the storage pool at I/O data rates within an I/O data rate limit defined by metadata associated with the storage pool, by;

receiving one or more storage access requests from the second virtual machine sent by the second virtual machine at a first I/O data rate that is faster than the I/O data rate limit defined in the metadata, and

sending the one or more storage access requests to the storage pool at a second I/O data rate, wherein the second I/O data rate is within the storage pool I/O data rate limit defined by the metadata.

12. The virtual storage area network of claim 11, further comprising:

configuring an operating system of the first virtual machine, wherein configuring the operating system of the first virtual machine includes configuring a network controller of the first virtual machine, and wherein the configured network controller of the first virtual machine dynamically modifies sending the one or more storage access requests to the storage pool at a I/O data rate approximately equal to the I/O data rate limit defined by the metadata.

13. The virtual storage area network of claim 11, further comprising:

traffic shaping the one or more storage access requests from operating at the first I/O data rate to operating at the second I/O data rate.

14. The virtual storage area network of claim 11, wherein the first virtual machine receives one or more storage access requests from a third virtual machine for accessing the storage pool, wherein the one or more storage access requests from the third virtual machine were sent at a third I/O data rate, and wherein the sum of the second I/O data rate and the third I/O data rate are greater than the storage pool I/O data rate limit defined by the metadata associated with the storage pool, wherein the one or more storage access requests from the second and the third virtual machines are throttled such that the one or more storage access requests are sent to the storage pool at I/O data rates within the storage pool I/O data rate limit defined by the metadata.

15. The virtual storage area network of claim 11, wherein the metadata defines one or more respective per-virtual machine I/O data rate limits different from and less than the storage pool I/O data rate limit for accessing the storage pool.

16. The virtual storage area network of claim 11, wherein in response to receiving a request to increase an amount of physical block storage space reserved at a physical block storage device for the storage pool, determining within the metadata whether the requested increase is within a maximum storage space limit associated with the storage pool, and reserving the requested increase at the physical block storage device if the increase of the storage space will not exceed the maximum storage space limit.

17. The virtual storage area network of claim 11, wherein the first virtual machine receives a request, via a graphical user interface, to change the storage pool I/O data rate limit for the storage pool, wherein the metadata associated with the storage pool is modified based on the request.

18. The virtual storage area network of claim 11, wherein the first virtual machine automatically decreases a per-virtual machine I/O data rate limit in response to determining that the per-virtual machine I/O data rate is at or near a threshold I/O data rate limit for the storage pool.

19. The virtual storage area network of claim 11, wherein the metadata determines a data storage limit for the storage pool.

20. The virtual storage area network of claim 11, wherein a logical file system is determined for the storage pool and wherein the logical file system includes a dataset for storing data accessible, via a network, to the second virtual machine.

21. A computer-implemented method for determining a virtual storage area network, the computer-implemented method comprising:

determining a virtual space capacity and a physical space capacity for a storage pool, wherein the storage pool is at least an allocation of data storage at a physical block storage device, wherein the physical space capacity determines the physical space reserved at a block storage device for the storage pool, and wherein the virtual space capacity determines an amount of data that a customer can store at the storage pool;

reserving physical block storage space at the physical block storage device for the storage pool, based on the storage pool'"'"'s physical space capacity; and

instantiating a host virtual machine configured to represent the storage pool as a mass storage device to other virtual machines.

22. The computer-implemented method of claim 21, wherein the host virtual machine is configured to forward for delivery to the storage pool one or more storage access requests sent from a virtual machine accessible to a virtual storage area customer.

23. The computer-implemented method of claim 21, wherein the host virtual machine is configured to forward storage access requests at an I/O data rate less than or equal to an I/O data rate limit for accessing the storage pool.

24. The computer-implemented method of claim 21, further comprising, in response to a customer request at a graphical user interface, increasing or decreasing the virtual space capacity for the storage pool by modifying a quota value associated with the storage pool.

25. The computer-implemented method of claim 21, further comprising increasing the storage pool'"'"'s physical space capacity by determining an additional storage pool at the physical block storage space for the virtual machine, wherein the additional storage pool is detected by the host machine as an additional mass storage device.

26. The computer-implemented method of claim 21, wherein the virtual space capacity is larger than the physical space capacity.

27. A computer-implemented virtual storage area network (VSAN), the computer-implemented VSAN comprising:

in a first virtual machine;

establishing a virtual storage transport connection between the first virtual machine and a physical block storage device;

designating a storage pool at the physical block storage device, wherein a storage pool is an allocation of data blocks for storing data and a logical file system;

representing the storage pool at the physical block storage device as a mass storage device;

exposing the reference to the mass storage device to a network to which the first virtual machine is connected;

receiving, based on the reference, a request from a second virtual machine to perform a block-level operation directed to data on a file system of the mass storage device; and

sending the request, via the virtual storage transport connection, to the physical block storage device for performing the block-level operation on data in the storage pool.

28. The computer-implemented VSAN of claim 27, further comprising configuring the virtual transport connection to limit the rate at which a request is performed.

29. A computer-implemented method for determining a virtual storage area network, the computer-implemented method comprising:

sending data from a first virtual machine to a storage pool at I/O data rates within an I/O data rate limit defined by metadata associated with the storage pool, by;

receiving one or more storage access requests from a second virtual machine sent by the second virtual machine at a first I/O data rate that is faster than the I/O data rate limit defined in the metadata, and

sending the one or more storage access requests to the storage pool at a second I/O data rate, wherein the second I/O data rate is within the storage pool I/O data rate limit defined by the metadata.

30. The computer-implemented method of claim 29, further comprising:

configuring an operating system of the first virtual machine, wherein configuring the operating system of the first virtual machine includes configuring a network controller of the first virtual machine, and wherein the configured network controller of the first virtual machine dynamically modifies sending the one or more storage access requests to the storage pool at a I/O data rate approximately equal to the I/O data rate limit defined by the metadata.

31. The computer-implemented method of claim 29, further comprising:

traffic shaping the one or more storage access requests from operating at the first I/O data rate to operating at the second I/O data rate.

32. The computer-implemented method of claim 29, wherein the first virtual machine receives one or more storage access requests from a third virtual machine for accessing the storage pool, wherein the one or more storage access requests from the third virtual machine were sent at a third I/O data rate, and wherein the sum of the second I/O data rate and the third I/O data rate are greater than the storage pool I/O data rate limit defined by the metadata associated with the storage pool, wherein the one or more storage access requests from the second and the third virtual machines are throttled such that the one or more storage access requests are sent to the storage pool at I/O data rates within the storage pool I/O data rate limit defined by the metadata.

33. The computer-implemented method of claim 29, wherein the metadata defines one or more respective per-virtual machine I/O data rate limits different from and less than the storage pool I/O data rate limit for accessing the storage pool.

34. The computer-implemented method of claim 29, wherein in response to receiving a request to increase an amount of physical block storage space reserved at a physical block storage device for the storage pool, determining within the metadata whether the requested increase is within a maximum storage space limit associated with the storage pool, and reserving the requested increase at the physical block storage device if the increase of the storage space will not exceed the maximum storage space limit.

35. The computer-implemented method of claim 29, wherein the first virtual machine receives a request, via a graphical user interface, to change the storage pool I/O data rate limit for the storage pool, wherein the metadata associated with the storage pool is modified based on the request.

36. The computer-implemented method of claim 29, wherein the first virtual machine automatically decreases a per-virtual machine I/O data rate limit in response to determining that the per-virtual machine I/O data rate is at or near a threshold I/O data rate limit for the storage pool.

37. The computer-implemented method of claim 29, wherein the metadata determines a data storage limit for the storage pool.

38. The computer-implemented method of claim 29, wherein a logical file system is determined for the storage pool and wherein the logical file system includes a dataset for storing data accessible, via a network, to the second virtual machine.

39. One or more computer memories collectively storing contents configured to cause a computer system to perform a method for determining a virtual storage area network, the method comprising:

determining a virtual space capacity and a physical space capacity for a storage pool, wherein the storage pool is at least an allocation of data storage at a physical block storage device, wherein the physical space capacity determines the physical space reserved at a block storage device for the storage pool, and wherein the virtual space capacity determines an amount of data that a customer can store at the storage pool;

reserving physical block storage space at the physical block storage device for the storage pool, based on the storage pool'"'"'s physical space capacity; and

instantiating a host virtual machine configured to represent the storage pool as a mass storage device to other virtual machines.

40. The computer memories of claim 39, wherein the host virtual machine is configured to forward for delivery to the storage pool one or more storage access requests sent from a virtual machine accessible to a virtual storage area customer.

41. The computer memories of claim 40, further comprising, in response to a customer request at a graphical user interface, increasing or decreasing the virtual space capacity for the storage pool by modifying a quota value associated with the storage pool.

42. The computer memories of claim 39, wherein the host virtual machine is configured to forward storage access requests at an I/O data rate less than or equal to an I/O data rate limit for accessing the storage pool.

43. The computer memories of claim 39, further comprising increasing the storage pool'"'"'s physical space capacity by determining an additional storage pool at the physical block storage space for the virtual machine, wherein the additional storage pool is detected by the host machine as an additional mass storage device.

44. The computer memories of claim 39, wherein the virtual space capacity is larger than the physical space capacity.

45. A computer-implemented method for determining a virtual storage area network (VSAN), the computer-implemented method comprising:

in a first virtual machine;

establishing a virtual storage transport connection between the first virtual machine and a physical block storage device;

designating a storage pool at the physical block storage device, wherein a storage pool is an allocation of data blocks for storing data and a logical file system;

representing the storage pool at the physical block storage device as a mass storage device;

exposing the reference to the mass storage device to a network to which the first virtual machine is connected;

receiving, based on the reference, a request from a second virtual machine to perform a block-level operation directed to data on a file system of the mass storage device; and

sending the request, via the virtual storage transport connection, to the physical block storage device for performing the block-level operation on data in the storage pool.

46. The computer-implemented method of claim 45, further comprising configuring the virtual transport connection to limit the rate at which a request is performed.