Multitenant hosted virtual machine infrastructure

• US 8,972,978 B2
• Filed: 05/02/2009
• Issued: 03/03/2015
• Est. Priority Date: 05/02/2008
• Status: Active Grant

• Current Assignee: Skytap Incorporated
• Original Assignee: Skytap Incorporated
• Inventors: Astete, Nicholas Luis, Brethorst, Aaron Benjamin, Goldberg, Joseph Michael, Hanlon, Matthew, Hutchinson, Anthony A. Jr., Janakiraman, Gopalakrishnan, Kotelnikov, Alexander, Novodvorskiy, Petr, Richardson, David William, Skelly, Roxanne Camille, Slioussar, Nikolai, Weeks, Jonathan
• Primary Examiner(s): Faherty, Corey S
• Application Number:
  US12/434,619
• Publication Number:
  US 20100138830A1
• Time in Patent Office: 2,131 Days
• Field of Search: None
• US Class Current: 718/1
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/455 Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0603 Catalogue ordering
• CPC Class Codes: G06Q 40/02 Banking, e.g. interest calculation or account maintenance credit or loans G06Q40/03
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources

Abstract

A multi-tenant virtual machine infrastructure (MTVMI) allows multiple tenants to independently access and use a plurality of virtual computing resources via the Internet. Within the MTVMI, different tenants may define unique configurations of virtual computing resources and unique rules to govern the use of the virtual computing resources. The MTVMI may be configured to provide valuable services for tenants and users associated with the tenants.

Claims

1. A method in a computing system including at least one processor for operating a virtual computing infrastructure for a plurality of tenants, comprising:

for each of at least a subset of the plurality of tenants,

receiving input specifying virtual computing resources from the virtual computing infrastructure to be used by users associated with the tenant;

allocating the specified virtual computing resources to a virtual data center for the tenant that operates in isolation from other virtual data centers operating in the virtual computing infrastructure;

receiving input from a privileged user associated with a distinguished tenant that specifies a plurality of users associated with the distinguished tenant;

receiving input from at least one user associated with the distinguished tenant that specifies;

a project to be established for the distinguished tenant that is associated with at least one of the specified plurality of users, and at least one of the following;

a virtual data center to be used for the specified project,

a virtual machine to be used for the specified project,

a lab to be used for the specified project, and

an asset to be to be used for the specified project; and

establishing for use by users of the specified plurality of users the specified project, together with any specified data centers, virtual machines, labs, and assets.

2. The method of claim 1 wherein, for at least one of the subset of the plurality of tenants, the received input specifies software for the virtual data center allocated to the tenant.

3. The method of claim 1, further comprising, for at least one of the subset of the plurality of tenants, receiving input specifying rules establishing rights of certain users to access virtual computing resources specified for the virtual data center for the tenant.

4. The method of claim 1 wherein, for at least one of the subset of the plurality of tenants, the received input specifies virtual hardware for the virtual data center allocated to the tenant.

5. The method of claim 1, further comprising, for at least one of the subset of the plurality of tenants:

providing users associated with the tenant access to the virtual data center via Internet website based access portals.

6. The method of claim 1, wherein the virtual computing resources comprise virtual machines that are instantiated on physical computing hardware not owned by any of the subset of the plurality of tenants.

7. The method of claim 1, wherein the virtual computing infrastructure executes on physical computing hardware administered by an entity separate from each of the subset of the plurality of tenants.

8. The method of claim 1, wherein the virtual computing resources comprise virtual machines that are instantiated by software executing on physical computing hardware where the software providing the virtual machines is not owned by any of the subset of the plurality of tenants.

9. The method of claim 1, wherein the virtual computing infrastructure is provided by software that executes on physical computing hardware where the software providing the virtual computing infrastructure is administered by an entity separate from each of the subset of the plurality of tenants.

10. The method of claim 1, further comprising:

receiving input from a user associated with a tenant specifying a virtual network services attribute for a virtual network incorporated into the virtual data center for the tenant; and

establishing the specified virtual network services attribute in the operation of the virtual data center, without effect on the operation of the other virtual data centers provided by the virtual computing infrastructure.

11. The method of claim 10 wherein the specified virtual network services attribute is selected from among:

a network address range for the virtual network, IP addresses for the virtual network, network interface (MAC) addresses for the virtual network, hostnames for the virtual network, and a VPN network tunnel that bridges the virtual network to another network.

12. The method of claim 1 wherein, for each tenant, the virtual data center comprises virtual network services that are isolated from virtual network services provided as part of other virtual data centers provided by the virtual computing infrastructure.

13. The method of claim 12 wherein the virtual network services that are isolated is selected from among:

name server, DHCP server, VPN server, firewall, NAT, traffic shaping, and load balancer.

14. The method of claim 1 wherein, for each tenant, the virtual data center comprises virtual storage services that are isolated from virtual storage services provided as part of other virtual data centers provided by the virtual computing infrastructure.

15. The method of claim 1, further comprising:

receiving a command from a user associated with a tenant identifying a subset of the virtual data center for the tenant; and

in response to the received command, persistently storing a complete state of the identified subset of the virtual data center for the tenant.

16. The method of claim 15 wherein the identified subset of the virtual data center is a proper subset of the virtual data center.

17. The method of claim 15 wherein the identified subset of the virtual data center is the entire virtual data center.

18. The method of claim 15, further comprising:

receiving an instantiation command from a user associated with the tenant; and

in response to the received instantiation command, using the stored complete state to create a new instance of the identified subset of the virtual data center for the tenant that is isolated from the virtual data center for the tenant a subset of whose state was persistently stored.

19. The method of claim 1, further comprising:

receiving a command from a user associated with a tenant identifying virtual computing resources of the virtual data center for the tenant; and

in response to the received command, deallocating the identified virtual computing resources.

20. The method of claim 1, further comprising:

receiving input from a privileged user associated with a distinguished tenant that specifies;

a plurality of users associated with the distinguished tenant, and

for each of the specified plurality of users, a role to be attributed to the user; and

for each of the specified plurality of users, providing to the user access to the virtual machines of the tenant based upon the role specified for the user.

21. The method of claim 1, further comprising:

receiving input from a privileged user associated with a distinguished tenant that specifies;

a plurality of users associated with the distinguished tenant, and

for each of the specified plurality of users, a role to be attributed to the user; and

for each of the specified plurality of users, providing to the user access to an entity of the tenant based upon the role specified for the user, the entity selected from among;

projects, assets, administering user privileges, resource quotas, resource usage reports, and audit trails.

22. The method of claim 21, wherein the user access provided to the entity of the tenant based upon the role specified user comprises a user access right selected from among:

read, use, change, delete, assign to project, upload, download, import, and export.

23. A method in a computing system including at least one processor for operating a virtual computing infrastructure for a plurality of tenants, comprising:

for each of at least a subset of the plurality of tenants,

receiving input specifying virtual computing resources from the virtual computing infrastructure to be used by users associated with the tenant;

allocating the specified virtual computing resources to a virtual data center for the tenant that operates in isolation from other virtual data centers operating in the virtual computing infrastructure; and

reporting to a user associated with a distinguished tenant resource usage both with respect to individual entities associated with the distinguished tenant and collectively for all of the individual entities associated with the distinguished tenant, wherein the resources whose usage is reported include total hours for which the virtual machines were executed individually by the entities and collectively for all of the entities.

24. The method of claim 23 wherein the individual entities are individual users associated with the distinguished tenant.

25. The method of claim 23 wherein the individual entities are individual projects associated with the distinguished tenant.

26. The method of claim 23 wherein the individual entities are individual virtual data centers associated with the distinguished tenant grouping subsets of the virtual machines instantiated for the distinguished tenant.

27. The method of claim 23 wherein the resources whose usage is reported include number of virtual machines controlled individually by the entities and collectively for all of the entities.

28. The method of claim 23 wherein the resources whose usage is reported include volume of storage individually by the entities and collectively for all of the entities.

29. The method of claim 23 wherein the resources whose usage is reported include number of public IP addresses individually by the entities and collectively for all of the entities.

30. The method of claim 23, wherein the resources whose usage is reported include a resource selected from among:

network bytes transferred, number of VPN tunnels, number of virtual machines imported, number of virtual machines exported, and number of users.

31. The method of claim 23, further comprising:

in connection with reporting resource usage to the user associated with the distinguished tenant, reporting to the user associated with the distinguished tenant a resource quota associated with the individual entities and a resource quota associated with the distinguished tenant.

32. The method of claim 31, further comprising:

when usage of a resource collectively for all of the individual entities associated with the distinguished tenant reaches the resource quota associated with the distinguished tenant, preventing further consumption of the resource by any of the individual entities associated with the distinguished tenant.

33. The method of claim 31, further comprising:

when usage of a resource by an individual entity associated with the distinguished tenant reaches the resource quota associated with the individual entity, preventing further consumption of the resource by the individual entity.

34. At least one tangible, computer-readable medium, having contents that when executed by at least one data processing device, performs a method, the method comprising:

for each of at least a subset of the plurality of tenants,

receiving input specifying virtual computing resources from the virtual computing infrastructure to be used by users associated with the tenant;

allocating the specified virtual computing resources to a virtual data center for the tenant that operates in isolation from other virtual data centers operating in the virtual computing infrastructure;

receiving input from a privileged user associated with a distinguished tenant that specifies a plurality of users associated with the distinguished tenant;

receiving input from at least one user associated with the distinguished tenant that specifies;

a project to be established for the distinguished tenant that is associated with at least one of the specified plurality of users, and at least one of the following;

a virtual data center to be used for the specified project,

a virtual machine to be used for the specified project,

a lab to be used for the specified project, and

an asset to be to be used for the specified project; and

establishing for use by users of the specified plurality of users the specified project, together with any specified data centers, virtual machines, labs, and assets.

35. The computer-readable medium of claim 34 wherein for at least one of the subset of the plurality of tenants, the received input specifies software for the virtual data center allocated to the tenant.

36. The computer-readable medium of claim 34 further comprising, for at least one of the subset of the plurality of tenants, receiving input specifying rules establishing rights of certain users to access virtual computing resources specified for the virtual data center for the tenant.

37. The computer-readable medium of claim 34 wherein for at least one of the subset of the plurality of tenants, the received input specifies virtual hardware for the virtual data center allocated to the tenant.

38. The computer-readable medium of claim 34 further comprising, for at least one of the subset of the plurality of tenants:

providing users associated with the tenant access to the virtual data center via Internet website based access portals.

39. The computer-readable medium of claim 34 wherein the virtual computing resources comprise virtual machines that are instantiated on physical computing hardware not owned by any of the subset of the plurality of tenants.

40. The computer-readable medium of claim 34 wherein the virtual computing infrastructure executes on physical computing hardware administered by an entity separate from each of the subset of the plurality of tenants.

41. The computer-readable medium of claim 34 wherein the virtual computing resources comprise virtual machines that are instantiated by software executing on physical computing hardware where the software providing the virtual machines is not owned by any of the subset of the plurality of tenants.

42. The computer-readable medium of claim 34 wherein the virtual computing infrastructure is provided by software that executes on physical computing hardware where the software providing the virtual computing infrastructure is administered by an entity separate from each of the subset of the plurality of tenants.

43. The computer-readable medium of claim 34 further comprising:

receiving input from a user associated with a tenant specifying a virtual network services attribute for a virtual network incorporated into the virtual data center for the tenant; and

establishing the specified virtual network services attribute in the operation of the virtual data center, without effect on the operation of the other virtual data centers provided by the virtual computing infrastructure.

44. The computer-readable medium of claim 43 wherein the specified virtual network services attribute is selected from among:

a network address range for the virtual network, IP addresses for the virtual network, network interface (MAC) addresses for the virtual network, hostnames for the virtual network, and a VPN network tunnel that bridges the virtual network to another network.

45. The computer-readable medium of claim 34 wherein, for each tenant, the virtual data center comprises virtual network services that are isolated from virtual network services provided as part of other virtual data centers provided by the virtual computing infrastructure.

46. The computer-readable medium of claim 45 wherein the virtual network services that are isolated is selected from among:

name server, DHCP server, VPN server, firewall, NAT, traffic shaping, and load balancer.

47. The computer-readable medium of claim 34 wherein, for each tenant, the virtual data center comprises virtual storage services that are isolated from virtual storage services provided as part of other virtual data centers provided by the virtual computing infrastructure.

48. The computer-readable medium of claim 34 further comprising:

receiving a command from a user associated with a tenant identifying a subset of the virtual data center for the tenant; and

in response to the received command, persistently storing a complete state of the identified subset of the virtual data center for the tenant.

49. The computer-readable medium of claim 48 wherein the identified subset of the virtual data center is a proper subset of the virtual data center.

50. The computer-readable medium of claim 48 wherein the identified subset of the virtual data center is the entire virtual data center.

51. The computer-readable medium of claim 48 further comprising:

receiving an instantiation command from a user associated with the tenant; and

in response to the received instantiation command, using the stored complete state to create a new instance of the identified subset of the virtual data center for the tenant that is isolated from the virtual data center for the tenant a subset of whose state was persistently stored.

52. The computer-readable medium of claim 34 further comprising:

receiving a command from a user associated with a tenant identifying virtual computing resources of the virtual data center for the tenant; and

in response to the received command, deallocating the identified virtual computing resources.

53. The computer-readable medium of claim 34 further comprising:

receiving input from a privileged user associated with a distinguished tenant that specifies;

a plurality of users associated with the distinguished tenant, and

for each of the specified plurality of users, a role to be attributed to the user; and

for each of the specified plurality of users, providing to the user access to the virtual machines of the tenant based upon the role specified for the user.

54. The computer-readable medium of claim 34 further comprising:

receiving input from a privileged user associated with a distinguished tenant that specifies;

a plurality of users associated with the distinguished tenant, and

for each of the specified plurality of users, a role to be attributed to the user; and

for each of the specified plurality of users, providing to the user access to an entity of the tenant based upon the role specified for the user, the entity selected from among;

projects, assets, administering user privileges, resource quotas, resource usage reports, and audit trails.

55. The method of claim 54, wherein the user access provided to the entity of the tenant based upon the role specified user comprises a user access right selected from among:

read, use, change, delete, assign to project, upload, download, import, and export.

56. At least one tangible, computer-readable medium, having contents that when executed by at least one data processing device, performs a method, the method comprising:

for each of at least a subset of the plurality of tenants,

receiving input specifying virtual computing resources from the virtual computing infrastructure to be used by users associated with the tenant;

allocating the specified virtual computing resources to a virtual data center for the tenant that operates in isolation from other virtual data centers operating in the virtual computing infrastructure; and

reporting to a user associated with a distinguished tenant resource usage both with respect to individual entities associated with the distinguished tenant and collectively for all of the individual entities associated with the distinguished tenant, wherein the resources whose usage is reported include total hours for which the virtual machines were executed individually by the entities and collectively for all of the entities.

57. The computer-readable medium of claim 56 wherein the individual entities are individual users associated with the distinguished tenant.

58. The computer-readable medium of claim 56 wherein the individual entities are individual projects associated with the distinguished tenant.

59. The computer-readable medium of claim 56 wherein the individual entities are individual virtual data centers associated with the distinguished tenant grouping subsets of the virtual machines instantiated for the distinguished tenant.

60. The computer-readable medium of claim 56 wherein the resources whose usage is reported include number of virtual machines controlled individually by the entities and collectively for all of the entities.

61. The computer-readable medium of claim 56 wherein the resources whose usage is reported include volume of storage individually by the entities and collectively for all of the entities.

62. The computer-readable medium of claim 56 wherein the resources whose usage is reported include number of public IP addresses individually by the entities and collectively for all of the entities.

63. The computer-readable medium of claim 56 wherein the resources whose usage is reported include a resource selected from among:

network bytes transferred, number of VPN tunnels, number of virtual machines imported, number of virtual machines exported, and number of users.

64. The computer-readable medium of claim 56 further comprising:

in connection with reporting resource usage to the user associated with the distinguished tenant, reporting to the user associated with the distinguished tenant a resource quota associated with the individual entities and a resource quota associated with the distinguished tenant.

65. The computer-readable medium of claim 64 further comprising:

when usage of a resource collectively for all of the individual entities associated with the distinguished tenant reaches the resource quota associated with the distinguished tenant, preventing further consumption of the resource by any of the individual entities associated with the distinguished tenant.

66. The computer-readable medium of claim 64 further comprising:

when usage of a resource by an individual entity associated with the distinguished tenant reaches the resource quota associated with the individual entity, preventing further consumption of the resource by the individual entity.