Event and state management in a scalable cloud computing environment

• US 8,856,319 B1
• Filed: 02/03/2011
• Issued: 10/07/2014
• Est. Priority Date: 02/03/2010
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Huang, Alex, Vittal, Chiradeep, Chan, Will, Liang, Sheng
• Primary Examiner(s): Srivastava, Vivek
• Assistant Examiner(s): Abedin, Normin
• Application Number:
  US13/020,803
• Time in Patent Office: 1,342 Days
• Field of Search: 709/224, 709/223, 709/225, 709/203, 709/226
• US Class Current: 709/224
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers

Abstract

A scalable cloud infrastructure serves two or more customers, where each customer is associated with at least one unit of virtual resources. The virtual resources are established by apportioning physical resources in the cloud infrastructure that are partitioned into pods within one or more zones in a scalable manner. Additionally, the cloud infrastructure establishes one or more management server clusters each comprising one or more management servers. The two or more customers create a number of virtual machines within pods in one or more zones. As customers interact primarily with the management server cluster to manage their virtual machine, a state and event handling method is devised to centralize the management of a potentially vast number of customer and supporting system virtual machines.

Claims

1. A method of reporting operating states for pluralities of virtual machines to a management server, the pluralities of virtual machines executing on respective compute servers coupled to the management server by a network, the pluralities of virtual machines being managed by respective agents executing on the respective physical compute servers, each operating state having a respective predetermined next operating state according to a predetermined state flow description of creation, use, and destruction of a virtual machine, the method comprising the steps, performed by each of the agents for the corresponding plurality of virtual machines managed by the agent, of:

obtaining and locally storing, for each of the virtual machines managed by the agent, first state information indicating a first operating state for the respective virtual machine; and

subsequently, during each of a plurality of successive sync intervals;

obtaining, for each of the virtual machines managed by the agent, second state information indicating a second operating state for the respective virtual machine;

determining, for each of the virtual machines managed by the agent, whether the second operating state of the virtual machine differs is a changed state differing from the first operating state of the virtual machine;

responsive to the determining indicating that the second operating state of the virtual machine is a changed state differing from the first operating state of the virtual machine, storing the second operating state for the virtual machine as a respective delta state; and

transmitting updated state information to the management server, the updated state information including only the delta states for the virtual machines having changed states and excluding the first state information for the virtual machines not having changed states.

2. The method of claim 1, the method further comprising storing the second operating state as the first state for the plurality of virtual machines for which the second operating state differs from the first operating state.

3. The method of claim 1, the method further comprising:

receiving a command specifying a desired operating state for a managed virtual machine; and

transitioning the virtual machine to the desired operating state.

4. The method of claim 1, wherein:

each compute server includes a hypervisor operative to monitor operating states of the respective virtual machines and to provide the first and second virtual machine information to the respective agent.

5. The method of claim 4, wherein each agent also issues control commands to the respective hypervisor to change an operating state of a virtual machine, the control commands including commands for starting a virtual machine and stopping a virtual machine.

6. The method of claim 5, wherein each agent issues the control commands in response to receiving corresponding management commands from the management server, the management commands resulting from one or more of customer-initiated events, administrator-initiated events and system-generated events communicated to the management server and requiring manipulation of the virtual machine.

7. The method of claim 1, wherein transmitting the delta states is part of a delta sync operation between the agent and the management server for communicating changes of operating states of the respective virtual machines, and further including a full sync operation in which the agent communicates all present operating states of all the respective virtual machines to the management server.

8. The method of claim 1, wherein the intervals include established time intervals and priority state change intervals, the established time intervals corresponding to successive requests from the management server to the agent for a delta sync, the priority state change intervals corresponding to successive delta syncs containing priority state changes being pushed by the agent to the management server.

9. The method of claim 8, wherein the priority state changes include unexpected deaths of virtual machines.

10. A non-transitory computer readable storage medium storing instructions executable by a compute server to perform a method of reporting operating states for a plurality of virtual machines to a management server coupled to the compute server by a network, the plurality of virtual machines executing on the compute serve and being managed by an agent executing on the compute server, each operating state having a respective predetermined next operating state according to a predetermined state flow description of creation, use, and destruction of a virtual machine, the method including the steps, performed by the agent for the plurality of virtual machines, of:

obtaining and locally storing, for each of the virtual machines, first state information indicating a first operating state for the respective virtual machine; and

subsequently, during each of a plurality of successive sync intervals;

obtaining and locally storing, for each of the virtual machines, second state information indicating a second operating state for the respective virtual machine;

determining, for each of the virtual machines, whether the second operating state of the virtual machine is a changed state differing from the first operating state of the virtual machine;

responsive to the determining indicating that the second operating state of the virtual machine is a changed state differing from the first operating state of the virtual machine, storing the second operating state for the virtual machine as a respective delta state; and

transmitting updated state information to the management server, the updated state information including only the delta states for the virtual machines having changed states and excluding the first state information for the virtual machines not having changed states.

11. The non-transitory computer readable storage medium of claim 10, further comprising instructions that cause the one or more processors to store the second operating state as the first operating state for the plurality of virtual machines for which the second operating state differs from the first operating state.

12. The non-transitory computer readable storage medium of claim 10, further comprising instructions that cause the one or more processors to:

receive a command specifying a desired operating state for a managed virtual machine; and

transition the virtual machine to the desired operating state.

13. A method of operating a management server to perform state management and management actions for a plurality of virtual machines executing on respective compute servers coupled to the management server by a network, the method comprising:

maintaining a record of an actual flow of operating states for each of the plurality of virtual machines, each operating state having a respective predetermined next valid operating state according to a state table reflecting an expected flow of operating states according to a predetermined state flow description of creation, use, and destruction of a virtual machine;

performing at the management server in response to receiving operating state updates for a portion of the virtual machines;

determining for the portion of the virtual machines a current operating state based on the record of operating states and the received operating state updates for each of the virtual machines in the portion; and

updating the record of operating states to reflect the current operating state for each of the portion of virtual machines; and

performing at the management server in response to receiving an event for a virtual machine;

accessing the record of operating states for the virtual machine to identify a current operating state therefor;

accessing the state table to identify the predetermined valid next operating state for current operating state indicated in the record of operating states for the virtual machine;

determining from the record of operating states and the state table whether the event is associated with a valid state transition from the current operating state to the predetermined valid next operating state; and

responsive to determining that the event is associated with the valid state transition, committing the event for processing and updating the record of operating states with the expected operating state, and otherwise failing the event.

14. The method of claim 13, wherein the record of operating states substantially comprises the first operating states determined previously at each of the plurality of agents.

15. The method of claim 13, wherein the record of operating states includes a revision number for determining the validity of the event.

16. The method of claim 15, wherein updating the record of operating states comprises updating the revision number, the revision number increased to indicate a committed transaction occurred.

17. The method of claim 13, wherein committing the event for processing comprises transmitting a command to perform an action to complete the event.

18. The method of claim 13, wherein the received event is determined at the management server from operating state updates for the portion of the virtual machines.

19. The method of claim 13, wherein the event is one of a customer-initiated event, an administrator-initiated event and a system-generated event communicated to the management server and requiring manipulation of the virtual machine.

20. The method of claim 13, wherein determining the validity of the event includes determining that the event commands a transition of an operating state of a virtual machine to the corresponding predetermined next operating state.

21. The method of claim 13, wherein the management server is a first management server having management responsibility for a first compute server on which the portion of the virtual machines are executing, and further including, upon migration of one of the virtual machines to a second compute server for which a second management server has management responsibility, transferring the record of operating state information to the second management server.

22. The method of claim 13, wherein the operating state updates are received as part of a delta sync operation between the management server and an agent for the portion of the virtual machines for communicating changes of the operating states of the respective virtual machines, and further including a full sync operation in which the management server receives all present operating states of all the portion of the virtual machines from the management agent.

23. A non-transitory computer readable storage medium storing instructions executable by a management server to perform a method of state management and management actions for a plurality of virtual machines executing on respective compute servers coupled to the management server by a network, by:

maintaining a record of an actual flow of operating states for each of the plurality of virtual machines, each operating state having a respective predetermined next valid operating state according to a state table reflecting an expected flow of operating states according to a predetermined state flow description of creation, use, and destruction of a virtual machine;

performing at the management server in response to receiving operating state updates for a portion of the virtual machines;

determining for the portion of the virtual machines a current operating state based on the record of operating states and the received operating state updates for each of the virtual machines in the portion; and

update the record of operating states to reflect the current operating state for each of the portion of virtual machines; and

performing at the management server in response to receiving an event for a virtual machine;

accessing the record of operating states for the virtual machine to identify a current operating state therefor;

accessing the state table to identify the predetermined valid next operating state for current operating state indicated in the record of operating states for the virtual machine;

determining from the record of operating states and the state table the validity of the event; and

responsive to determining that the event is associated with the valid state transition, committing the event for processing and updating the record of operating states with the expected operating state or and otherwise failing the event.

24. The non-transitory computer readable storage medium of claim 23, wherein the record of operating states substantially comprises the first operating states determined previously at each of the plurality of agents.

25. The non-transitory computer readable storage medium of claim 23, wherein the record of operating states includes a revision number for determining the validity of the event.

26. The non-transitory computer readable storage medium of claim 25, wherein updating the record of operating state information comprises updating the revision number, the revision number increased to indicate a committed transaction occurred.

27. The non-transitory computer readable storage medium of claim 23, wherein committing the event for processing comprises transmitting a command to perform an action to complete the event.

28. The non-transitory computer readable storage medium of claim 23, wherein the received event is determined at the management server from operating state updates for the portion of the virtual machines.