Multitenant hosted virtual machine infrastructure

• US 10,127,059 B2
• Filed: 05/02/2009
• Issued: 11/13/2018
• Est. Priority Date: 05/02/2008
• Status: Active Grant

• Current Assignee: Skytap Incorporated
• Original Assignee: Skytap Incorporated
• Inventors: Astete, Nicholas Luis, Brethorst, Aaron Benjamin, Goldberg, Joseph Michael, Hanlon, Matthew, Hutchinson, Jr., Anthony A., Janakiraman, Gopalakrishnan, Kotelnikov, Alexander, Novodvorskiy, Petr, Richardson, David William, Skelly, Roxanne Camille, Slioussar, Nikolai, Weeks, Jonathan
• Primary Examiner(s): Faherty, Corey S
• Application Number:
  US12/434,621
• Publication Number:
  US 20090288084A1
• Time in Patent Office: 3,482 Days
• Field of Search: None
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: G06Q 40/02 Banking, e.g. interest calculation or account maintenance credit or loans G06Q40/03
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources

Abstract

A multi-tenant virtual machine infrastructure (MTVMI) allows multiple tenants to independently access and use a plurality of virtual computing resources via the Internet. Within the MTVMI, different tenants may define unique configurations of virtual computing resources and unique rules to govern the use of the virtual computing resources. The MTVMI may be configured to provide valuable services for tenants and users associated with the tenants.

Claims

1. A plurality of hardware computer systems each including at least one processor, comprising:

a virtual machine management module executing on at least one of the processors that causes a plurality of virtual machines to be executed on at least one of the processors in accordance with instructions from each of a plurality of principals;

a plurality of persistent storage devices directly or indirectly accessible, via one or more physical storage controllers, to the processors on which the plurality of virtual machines are executed; and

virtual storage management software executed on at least one of the processors to;

receive a persistent storage request from one of the plurality of virtual machines, and

in response to receiving the persistent storage request from one of the plurality of virtual machines;

dynamically choosing a proper subset of the plurality of persistent storage devices to service the persistent storage request received from one of the plurality of virtual machines, wherein the subset of the plurality of persistent storage device has two or more persistent storage devices, and

dynamically selecting a respective persistent storage device of the identified subset to service the persistent storage request.

2. The plurality of hardware computer systems of claim 1 wherein, for persistent storage requests that are read requests specifying data to be read, the identified subset includes any of the plurality of persistent storage devices on which the specified data to be read is stored.

3. The plurality of hardware computer systems of claim 1 wherein, for each persistent storage request that is a write request that has a relationship to already-stored data, the identified subset includes any of the plurality of persistent storage devices on which the already-stored data is stored.

4. The plurality of hardware computer systems of claim 3 wherein at least one request that has a relationship to already stored data is a copy request.

5. The plurality of hardware computer systems of claim 3 wherein at least one request that has a relationship to already stored data is a snapshot request.

6. The plurality of hardware computer systems of claim 1 wherein a persistent storage device of the identified subset is selected based upon the relative health of the persistent storage devices of the identified subset.

7. The plurality of hardware computer systems of claim 1 wherein the virtual machines caused to be executed by the virtual machine management module are each executed on behalf of one of the plurality of principals,

and wherein at least a portion of the plurality of persistent storage devices of the identified subset have been assigned an affinity for a particular principal of the plurality of principals,

and wherein a persistent storage device of the identified subset is selected that has been assigned an affinity for the principal on behalf of which the virtual machine or the virtual machine management module making the persistent storage request is executed.

8. A computer-implemented method in a multi-tenant virtual machine infrastructure system including one or more processors, comprising:

causing a plurality of virtual machines to be executed on at least one of the one or more processors in accordance with instructions from each of a plurality of principals, wherein a plurality of persistent storage devices are directly or indirectly accessible, via one or more physical storage controllers;

receiving, in the multi-tenant virtual machine infrastructure, a first persistent storage request and a second persistent storage request from a first virtual machine of the plurality of virtual machines;

dynamically choosing a proper subset of the plurality of persistent storage devices to service the first persistent storage request and the second persistent storage request received from the first virtual machine, wherein the subset of the plurality of persistent storage devices has two or more persistent storage devices;

delegating responsibility for servicing the first persistent storage request to a first persistent storage device of the identified subset of the plurality of persistent storage devices; and

delegating responsibility for servicing the second persistent storage request to a second persistent storage device of the identified subset of the plurality of persistent storage devices that is distinct from the first persistent storage device.

9. The method of claim 8 wherein the delegating comprises:

identifying a proper subset of the plurality of persistent storage devices that are candidates for selection; and

selecting a persistent storage device of the identified subset to service the persistent storage request.

10. The method of claim 9 wherein, for persistent storage requests that are read requests specifying data to be read, the identified subset includes any of the plurality of persistent storage devices on which the specified data to be read is stored.

11. The method of claim 9 wherein, for each persistent storage request that is a write request that has a relationship to already-stored data, the identified subset includes any of the plurality of persistent storage devices on which the already-stored data is stored.

12. The method of claim 9 wherein the virtual machines are each executed on behalf of one of the plurality of principals,

and wherein at least a portion of the plurality of persistent storage devices of the identified subset have been assigned an affinity for a particular principal of the plurality of principals,

and wherein a persistent storage device of the identified subset is selected that has been assigned an affinity for the principal on behalf of which the virtual machine making the persistent storage request is executed.

19. The computer-implemented method of claim 8, further comprising, in response to a third storage request for data replicated across each of the plurality of persistent storage devices, determining a subset of the plurality of persistent storage devices to retrieve the data, wherein the subset is determined based on content data, capacity data, usage data, performance data, load data, health data or affinity data, or a combination thereof, obtained from each of the plurality of persistent storage devices.

13. A plurality of hardware computer systems each including at least one processor, comprising:

a virtual machine management means for causing a plurality of virtual machines to be executed on at least one of the processors in accordance with instructions from each of a plurality of principals; and

a virtual storage management means executed on at least one of the processors to;

receive a request from each of the plurality of virtual machines, and

determine for each persistent storage request received by the virtual machine management means, which of a plurality of persistent storage devices should service the persistent storage request, wherein determining includes;

choosing a subset of the plurality of persistent storage devices to service each persistent storage request, wherein the subset of the plurality of persistent storage devices has two or more persistent storage devices, and

selecting a respective persistent storage device of the identified subset to service the persistent storage request.

14. At least one computer-readable storage device which is not an electro-magnetic signal having contents adapted to cause a data processing device to perform a method for servicing a persistent storage request, the method comprising:

causing virtual machines to be executed on at least one of the data processing devices in accordance with instructions from each of a plurality of principals, wherein a plurality of persistent storage devices are directly or indirectly accessible, via one or more physical storage controllers, to the data processing device on which the virtual machines are executed;

causing virtual storage management software configured to control storage operations of virtual machines to be executed on at least one of the data processing devices to perform the functions of;

receiving, in the multi-tenant virtual machine infrastructure, one or more persistent storage requests from at least one of the virtual machines; and

in response to receiving the one or more persistent storage requests, dynamically choosing a subset of the plurality of persistent storage devices for servicing the at least persistent storage request, based on one or more storage node characteristics, wherein the subset of the plurality of persistent storage devices has two or more persistent storage devices; and

sending the at least one or more persistent storage request for delivery to a respective persistent storage device of the identified subset responsible for servicing the corresponding request.

15. The computer-readable storage device of claim 14 wherein the allocating responsibility includes:

identifying a proper subset of the plurality of persistent storage devices that are candidates for selection; and

selecting a persistent storage device of the identified subset to service the persistent storage request.

16. The computer-readable storage device of claim 14 wherein, for persistent storage requests that are read requests specifying data to be read, the identified subset includes any of the plurality of persistent storage devices on which the specified data to be read is stored.

17. The computer-readable storage device of claim 14 wherein, for each persistent storage request that is a write request that has a relationship to already-stored data, the identified subset includes any of the plurality of persistent storage devices on which the already-stored data is stored.

18. The computer-readable storage device of claim 14 wherein the virtual machines are each executed on behalf of one of the plurality of principals,

and wherein at least a portion of the plurality of persistent storage devices of the identified subset have been assigned an affinity for a particular principal of the plurality of principals,

and wherein a persistent storage device of the identified subset is selected that has been assigned an affinity for the principal on behalf of which the virtual machine making the persistent storage request is executed.

20. The computer-implemented method of claim 14, further comprising dynamically and periodically obtaining the one or more storage node characteristics from each of the plurality of persistent storage devices, wherein dynamically delegating responsibility for servicing the first persistent storage request and the second persistent storage request is based on at least one of the storage-related characteristics, wherein storage -related characteristics include capacity data, usage data, load data, health data, or affinity data, or a combination thereof.