Virtual private storage array service for cloud servers

• US 8,819,230 B2
• Filed: 11/05/2011
• Issued: 08/26/2014
• Est. Priority Date: 11/05/2011
• Status: Active Grant

• Current Assignee: Zadara Storage Limited
• Original Assignee: Zadara Storage Limited
• Inventors: Nahum, Nelson, Hershko, Yair, Ossia, Yoav, Vainblat, Lev, Lyakas, Alex, Melnikov, Moshe, Popovski, Vladimir, V., Shyam Kaushik
• Primary Examiner(s): BOUTAH, ALINA A
• Application Number:
  US13/290,084
• Publication Number:
  US 20130117448A1
• Time in Patent Office: 1,025 Days
• Field of Search: 709225-226
• US Class Current: 709/225
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources

Abstract

A method for providing virtual private storage array (VPSA) service for cloud users over a computer network includes receiving parameters for the VPSA over the network and creating the VPSA from resources of server computers. Creating the VPSA includes allocating and exposing drives that meets or exceeds specified drive characteristics, drive quantity, and array redundancy criteria to virtual controllers (VCs) in the VPSA, and dedicating parts of processor/memory complexes that each meets or exceeds a specified virtual controller hardware model to the VCs. The VCs run on virtual machines on the dedicated parts of processor/memory complexes on independent server computers. The VCs discover the exposed drives, create a virtual pool from the exposed virtual drives, implement data protection on the virtual pool, create volumes from the virtual pool, expose the volumes over the network to a customer computer, and handle access requests to the volumes from the customer computer.

Claims

1. A method to provide virtual private storage array service over a computer network for cloud servers in a public or a private cloud, comprising:

receiving parameters for a virtual private storage array over the computer network, the parameters for the virtual private storage array including a virtual controller hardware model for each virtual controller in the virtual private storage array, drive characteristics for the virtual private storage array, and a drive quantity for the virtual private storage array;

creating the virtual private storage array from processor/memory complexes and physical drives available from server computers;

creating one or more volumes from the virtual private storage array;

exposing the one or more volumes over the computer network to one or more cloud servers; and

handling access requests to the exposed one or more volumes over the computer network from the one or more cloud servers;

wherein;

each server computer runs software including at least one of a storage node and a compute agent; and

said creating the virtual private storage array, said creating one or more volumes, said exposing the one or more volumes, and said handling access requests comprise;

selecting a set of the physical drives that meets or exceeds the drive characteristics and the drive quantity, the selected physical drives being from a set of the server computers;

instructing storage nodes on the set of the server computers to allocate virtual drives to the virtual private storage array, the storage nodes being configured to;

create the virtual drives from the selected physical drives, each virtual drive being a partition that is one entire selected physical drive or a part of one selected physical drive; and

expose the virtual drives to virtual controllers in the virtual private storage array;

selecting a set of the processor/memory complexes that each meets or exceeds the virtual controller hardware model, the selected processor/memory complexes being from an other set of the server computers;

instructing compute agents on the other set of the server computers to spawn virtual machines for the virtual controllers, the compute agents being configured to;

spawn one virtual machine on at least part of each selected processor/memory complex dedicated to the virtual machine; and

start one virtual controller per virtual machine so each virtual controller in the virtual private storage array runs on a different server computer, one or more of the virtual controllers being configured to;

 

discover the exposed virtual drives;

 

create one or more virtual pools comprising the exposed virtual drives;

 

implement a data protection scheme on the one or more virtual pools;

 

create the one or more volumes from the one or more virtual pools;

 

expose the one or more volumes over the computer network to the one or more cloud servers; and

 

handle the access requests to the exposed one or more volumes over the computer network from the one or more cloud servers.

2. The method of claim 1, wherein:

the storage nodes are further configured to;

discover the physical drives;

query the physical drives for drive types and drive capacities; and

transmit the drive types and the drive capacities to an availability zone controller;

said selecting a set of the physical drives comprises;

creating a list of candidate server computers with physical drives meeting or exceeding the drive characteristics;

sorting the list according to one or more sorting criteria; and

selecting physical drives of top ranking candidate server computers.

3. The method of claim 1, wherein said selecting a set of the processor/memory complexes comprise:

querying the server computers for the processor/memory complexes;

creating a list of candidate server computers with processor/memory complexes meeting or exceeding the virtual controller hardware model;

sorting the list according to one or more sorting criteria; and

selecting processor/memory complexes of top ranking candidate server computers.

4. The method of claim 1, wherein the set of server computers includes part or all of the server computers.

5. The method of claim 1, wherein the set of the server computers and the other set of the server computers overlap.

6. The method of claim 1, wherein said handling access requests includes authenticating the access requests to ensure data privacy.

7. A method to provide a virtual private storage array as a service over a computer network, the method comprising:

receiving parameters for the virtual private storage array over the computer network, the parameters for the virtual private storage array including a virtual controller hardware model for each virtual controller in the virtual private storage array, drive characteristics for the virtual private storage array, and a drive quantity for the virtual private storage array; and

creating the virtual private storage array from processor/memory complexes and physical drives available from server computers, each server computer running software including at least one of a storage node and a compute agent, said creating the virtual private storage array comprising;

selecting a set of the physical drives that meets or exceeds the drive characteristics and the drive quantity, the selected physical drives being from a set of the server computers;

instructing storage nodes on the set of the server computers to allocate virtual drives to the virtual private storage array, the storage nodes being configured to;

create the virtual drives from the selected physical drives, each virtual drive being a partition that is one entire selected physical drive or a part of one selected physical drive; and

expose the virtual drives to virtual controllers in the virtual private storage array;

selecting a set of the processor/memory complexes that each meets or exceeds the virtual controller hardware model, the selected processor/memory complexes being from an other set of the server computers;

instructing compute agents on the other set of the server computers to spawn virtual machines for the virtual controllers, the compute agents being configured to;

spawn one virtual machine on at least part of each selected processor/memory complex dedicated to the virtual machine; and

start one virtual controller per virtual machine so each virtual controller in the virtual private storage array runs on a different server computer, one or more of the virtual controllers being configured to;

discover the exposed virtual drives;

create one or more virtual pools comprising the exposed virtual drives;

implement a data protection scheme on the one or more virtual pools;

create one or more volumes from the one or more virtual pools;

expose the one or more volumes over the computer network to one or more customer computers; and

handle access requests to the exposed one or more volumes over the computer network from the one or more customer computers.

8. The method of claim 7, further comprising:

transmitting a web form over the computer network, the web form being configured to pass back the parameters;

determining a service fee for the virtual private storage array;

transmitting a web page over the computer network, the web page including the service fee, the web page being configured to pass back a confirmation to create the virtual private storage array; and

receiving the confirmation over the computer network from the customer computer.

9. The method of claim 7, wherein:

the virtual controller hardware model specifies at least one of a central processing unit model, a central processing unit feature, a central processing unit quantity, a random access memory capacity, and a networking bandwidth; and

the drive characteristics specify at least one of a drive type, a drive capacity, and a drive encryption.

10. The method of claim 7, wherein:

each virtual drive is a logical volume created from a partition that is an entire physical drive or part of a physical drive; and

the storage nodes are configured to expose the virtual drives to the virtual controllers by exposing the virtual drives as network block devices to the virtual controllers.

11. The method of claim 7, wherein the parameters further include one of a RAID level of data protection for the virtual private storage array and a virtual controller quantity in the virtual private storage array.

12. The method of claim 7, wherein:

the storage nodes are further configured to;

discover the physical drives;

query the physical drives for drive types and drive capacities; and

transmit the drive types and the drive capacities to an availability zone controller;

said selecting a set of the physical drives comprises;

creating a list of candidate server computers with physical drives meeting or exceeding the drive characteristics;

sorting the list according to one or more sorting criteria; and

selecting physical drives of top ranking candidate server computers.

13. The method of claim 12, wherein the one or more sorting criteria include utilization rates of the candidate server computers.

14. The method of claim 7, wherein said selecting a set of the processor/memory complexes comprise:

querying the server computers for the processor/memory complexes;

creating a list of candidate server computers with processor/memory complexes meeting or exceeding the virtual controller hardware model;

sorting the list according to one or more sorting criteria; and

selecting processor/memory complexes of top ranking candidate server computers.

15. The method of claim 14, wherein the one or more sorting criteria include utilization rates of the candidate server computers.

16. The method of claim 7, wherein the virtual controllers are configured for load-sharing and redundancy.

17. The method of claim 7, wherein the set of server computers includes part or all of the server computers.

18. The method of claim 7, wherein the set of the server computers and the other set of the server computers overlap.

19. The method of claim 7, wherein the physical drives include one or more of magnetic hard disk drives, solid state drives, and phase-change memory drives.

20. The method of claim 7, wherein the one or more virtual controllers are configured to handle the access requests by authenticating the access requests to ensure data privacy.

21. A data storage system providing a virtual private storage array as a service over a computer network in a cloud computing environment to one or more customer computers in a public or a private cloud, the data storage system comprising:

an availability zone controller being configured to create the virtual private storage array from processor/memory complexes and physical drives available from server computers by;

receiving parameters for the virtual private storage array, the parameters including a virtual controller hardware model for each virtual controller in the virtual private storage array, drive characteristics for the virtual private storage array, and a drive quantity for the virtual private storage array;

selecting a set of the physical drives that meets or exceeds the drive characteristics and the drive quantity, the selected physical drives being from a set of the server computers;

instructing storage nodes on the set of the server computers to allocate virtual drives from the selected physical drives to the virtual private storage array;

selecting a set of the processor/memory complexes that each meets or exceeds the virtual controller hardware model, the selected processor/memory complexes being from an other set of the server computers; and

instructing compute agents on the other set of the server computers to spawn virtual machines for virtual controllers;

the server computers with the physical drives and the processor/memory complexes, the server computers including at least one server computer comprising;

one or more physical drives;

a processor/memory complex;

a storage node running on a first part of the processor/memory complex, the storage node being configured to;

create one or more virtual drives from the one or more physical drives, each virtual drive being a partition that is one entire selected physical drive or a part of one selected physical drive; and

expose the one or more virtual drives to the virtual controllers in the virtual private storage array;

a compute agent being configured to;

spawn a virtual machine on a second part of the processor/memory complex dedicated to the virtual machine; and

start a virtual controller on the virtual machine;

the virtual machine running on the second part of the processor/memory complex dedicated to the virtual machine;

the virtual controller running on the virtual machine, the virtual controller being configured to;

discover the one or more exposed virtual drives;

create one or more virtual pools comprising the exposed one or more virtual drives;

implement a data protection scheme on the one or more virtual pools;

create one or more volumes from the one or more virtual pools;

expose the one or more volumes over the computer network to the one or more customer computers; and

handle access requests to the exposed one or more volumes over the computer network from the one or more customer computers;

one or more public switches coupling the computer network and the other set of the server computers for communication between the virtual controllers and the one or more customer computers; and

one or more private switches coupling the set of the server computers for communication between the virtual controllers and the storage nodes located on different server computers.

22. The system of claim 21, further comprising a web server configured to:

transmit a web form over the computer network to a customer computer, the web form being configured to pass back the parameters;

receiving the parameters over the computer network and transmitting the parameters to the availability zone controller;

transmitting a web page over the computer network, the web page including a service fee for the virtual private storage array, the web page being configured to pass back a confirmation to create the virtual private storage array; and

receiving the confirmation over the computer network and transmitting the confirmation to the availability zone controller.

23. The system of claim 21, wherein:

the virtual controller hardware model specifies at least one of a central processing unit model, a central processing unit feature, a central processing unit quantity, a random access memory capacity, and a networking bandwidth; and

the drive characteristics specify at least one of a drive type, a drive capacity, and a drive encryption.

24. The system of claim 21, wherein:

each virtual drive is a logical volume created from a partition that is an entire physical drive or part of a physical drive; and

the storage node is configured to expose the one or more virtual drives to the virtual controllers by exposing the one or more virtual drives as one or more network block devices to the virtual controllers.

25. The system of claim 21, wherein the parameters further include one of a RAID level of data protection for the virtual private storage array and a virtual controller quantity in the virtual private storage array.

26. The system of claim 21, wherein:

the storage node is further configured to;

discover the one or more physical drives;

query the one or more physical drives for drive types and drive capacities; and

transmit the drive types and the drive capacities to the availability zone controller; and

the availability zone controller is configured to select the set of the physical drives by;

creating a list of candidate server computers with physical drives meeting or exceeding the drive characteristics;

sorting the list according to one or more sorting criteria; and

selecting physical drives of top ranking candidate server computers.

27. The system of claim 26, wherein the one or more sorting criteria include utilization rates of the candidate server computers.

28. The system of claim 21, wherein the availability zone controller is configured to select the set of the processor/memory complexes by:

querying the server computers for the processor/memory complexes;

creating a list of candidate server computers with processor/memory complexes meeting or exceeding the virtual controller hardware model;

sorting the list according to one or more sorting criteria; and

selecting processor/memory complexes of top ranking candidate server computers.

29. The system of claim 28, wherein the one or more sorting criteria include utilization rates of the candidate server computers.

30. The system of claim 21, wherein the virtual controllers are configured for load-sharing and redundancy.

31. The system of claim 21, wherein the set of the server computers include part or all of the server computers.

32. The system of claim 21, wherein the set of the server computers and the other set of the server computers overlap.

33. The system of claim 21, wherein the physical drives include one or more of magnetic hard disk drives, solid state drives, and phase-change memory drives.

34. The system of claim 21, wherein the virtual controllers are configured to handle the access requests by authenticating the access requests to ensure data privacy.