Systems and methods for path-based management of virtual servers in storage network environments

• US 8,560,671 B1
• Filed: 09/09/2008
• Issued: 10/15/2013
• Est. Priority Date: 10/23/2003
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Yahalom, Raphael, Levy, Assaf, Kessel, Omri, Alon, Roee, Yaakobi, Michael
• Primary Examiner(s): Hussain, Tauqir
• Application Number:
  US12/283,163
• Time in Patent Office: 1,862 Days
• Field of Search: 709/224
• US Class Current: 709/224
• CPC Class Codes: G06F 11/3442 for planning or managing the needed capacity
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: G06F 9/45545 Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

Systems and methods for analyzing the service and performance levels associated with virtual machines in a storage network environment for compliance with a resource capacity policy are provided. Component configuration and connectivity information from components in the network environment is collected without using host agents on the virtual machines. Access paths defining end-to-end access relationships between an application on a virtual machine and storage data objects associated with the virtual machine in the network environment are derived. Access paths comprise sequences of components configured to enable information flow between an application residing on a virtual machine and a data object on a respective storage device. Access path resource consumption is computed and virtual machines with resource consumptions that violate the resource capacity policy are identified.

Claims

1. A process for analyzing virtual machines in a storage network environment for compliance with a resource capacity policy comprising:

collecting, without use of a host agent operating on the virtual machines, component configuration and connectivity information from components in the network environment;

deriving access paths defining end-to-end access relationships between an application on a virtual machine and storage data objects associated with the virtual machine in the network environment, wherein an access path includes a sequence of components enabling information flow between an application residing on a virtual machine and a data object, wherein deriving access paths comprises mapping network component configuration and connectivity information to nodes and edges of a graph and determining whether information can flow between nodes in the graph, wherein a node in the graph represents a component in the storage network environment, and an edge represents information flow capability between two components, wherein the information flow capability is determined by a physical communication link between the two components and the logical state configurations of the two components;

computing access path resource consumption and determining a respective rank classification of a virtual machine on the respective access path based on a level of storage service supported by the virtual machine; and

identifying virtual machines with resource consumptions and respective rank classifications that violate the resource capacity policy.

2. The process of claim 1, wherein the resource capacity policy includes a list of virtual machines in the storage network environment and the storage capacity allocated to each of the virtual machines.

3. The process of claim 1, wherein the access paths have access path attributes including at least one of a number of hops within an access path, a level of end-to-end redundancy for an access path, and a number of allocated ports for an access path.

5. The process of claim 1, further comprising of periodically generating summary reports about the virtual machines in the network containing:

a current amount of allocated and reserved virtual machine resources;

a ratio of the amount of allocated and reserved virtual machine resources to an amount of available virtual machine resources in the corresponding component, or to an amount of allocated and reserved virtual machine resources at other components of the same type and tier; and

a cost of the allocated or reserved virtual machine resources.

28. The process of claim 1, wherein the respective rank classification for the virtual machine on the respective access path represents a rank classification relative to other virtual machines in the storage network environment.

4. A process for analyzing virtual machines in a storage network environment for compliance with a resource capacity policy comprising:

collecting, without use of a host agent operating on the virtual machines, component configuration and connectivity information from components in the network environment;

deriving access paths defining end-to-end access relationships between an application on a virtual machine and storage data objects associated with the virtual machine in the network environment, wherein an access path includes a sequence of components enabling information flow between an application residing on a virtual machine and a data object;

computing access path resource consumption and determining a respective rank classification of a virtual machine on the respective access path based on a level of storage service supported by the virtual machine;

identifying virtual machines with resource consumptions and respective rank classifications that violate the resource capacity policy;

accumulating for an application the amount of allocated and reserved virtual machine resources allocated to the application and classifying the allocated and reserved virtual machine resources into tiers and into allocated and reserved virtual machine resources on the respective access paths and allocated and reserved virtual machine resources not on a respective access path;

monitoring the network environment and the resource capacity policy;

detecting virtual machine state change events and obtaining updated virtual machine state information;

using the updated virtual machine state information to compute for an application an accumulated amount of allocated and reserved virtual machine resources allocated to the application and their rank classifications; and

using the updated state information to analyze the allocated and reserved virtual machine resources allocated to or reserved for the application and determine whether the allocated and reserved virtual machine resources are currently accessible by the application via an access path associated with the virtual machines.

6. A process for analyzing virtual machines in a storage network environment for compliance with a resource capacity policy, comprising:

collecting, without use of a host agent operating on the virtual machines, component configuration and connectivity information from components in the network environment;

deriving access paths defining end-to-end access relationships between an application on a virtual machine and storage data objects associated with the virtual machine in the network environment, wherein an access path includes a sequence of components enabling information flow between an application residing on a virtual machine and a data object, and wherein the access paths have access path attributes, said access path attributes including at least one of a number of hops within an access path, a level of end-to-end redundancy for an access path, and a number of allocated ports for an access path;

using the component configuration and connectivity information to construct a graph to derive access paths by determining whether information can flow between nodes in the graph, wherein nodes in the graph represent components in the storage network environment, and edges represent information flow capability between two components, and wherein the information flow capability is determined by a physical communication link between the two components and the logical state configurations of the two components;

computing access path resource consumption and a respective rank classification of a virtual machine on a respective access path based on a level of storage service supported by the virtual machine; and

identifying virtual machines by comparing the constructed graph to the resource capacity policy to identify violations.

7. The process of claim 6, wherein the resource capacity policy includes a list of virtual machines in the storage network environment and the storage capacity allocated to each of the virtual machines.

29. The process of claim 6, wherein the respective rank classification for the virtual machine on the respective access path represents a rank classification relative to other virtual machines in the storage network environment.

8. A process for responding to a state change request for a virtual machine in a storage network environment comprising:

selecting virtual machine resources for satisfying the request;

deriving access paths associated with the respective virtual machines, wherein an access path includes a sequence of components enabling information flow between an application residing on a virtual machine and a data object on a respective storage device;

computing access path attributes for the access paths associated with the selected virtual machines;

planning allocation of the virtual machine resources;

reserving virtual machine resources of a selected type and a selected rank classification based on a level of storage service supported by the respective virtual machines;

allocating each reserved virtual machine resource;

making additional virtual machine resources available;

validating the request has been satisfied; and

notifying designated users of the validation.

9. The process of claim 8, wherein reserving a virtual machine resource comprises checking that the virtual machine resource is not already reserved;

checking that the virtual machine resource is not already allocated;

informing the reservation repository about the virtual machine resource and reservation attributes, including an associated application, a reservation timestamp, a reservation user, and constraints associated with converting the reserved virtual machine resource to an allocated virtual machine resource.

10. The process of claim 8, wherein allocating virtual machine resources comprises:

checking that a virtual machine resource is not already reserved by another application;

checking that the virtual machine resource is not already allocated;

informing the reservation repository about the virtual machine resource and allocation attributes, including an associated application, an allocation timestamp, and an allocation user; and

updating the state of a corresponding environment component to associate the reserved virtual machine resource with a corresponding application.

11. The process of claim 8, wherein making additional virtual machine resources available comprises:

informing the reservation repository that the virtual machine resources are available and updating the state of the corresponding environment component to disassociate the corresponding virtual machines from any application.

30. The process of claim 8, wherein the selected rank classification for the reserved virtual machine resources represents a rank classification relative to other virtual machine resources in the storage network environment.

12. A system for managing the capacity of virtual machine resources associated with applications in a storage network environment, comprising:

a user interface;

a display;

a memory for storing computer-executable instructions;

a processor in communication with the storage network environment, the processor configured for;

collecting information from sources in the network environment and deriving access paths defining end-to-end access relationships between an application on a virtual server and data objects associated with the virtual server in the network environment, wherein an access path includes a sequence of components configured to enable information flow between an application residing on a virtual machine and a data object on a respective storage device;

establishing rank classifications representative of the importance of a virtual machine resource in the network, and based on a level of storage service supported by the virtual machine;

computing for an access path associated with respective virtual machines the amount of its allocated and reserved virtual machine resources and the corresponding rank classifications;

collecting information about virtual machine resources from a virtual machine management platform and rank classifications reserved for a host application from a storage management platform;

accumulating for an application the amount of allocated and reserved virtual machine resources allocated to the application and classifying allocated and reserved virtual machine resources tiers and into allocated and reserved virtual machine resources on access paths and allocated and reserved virtual machine resources not on access paths;

constructing a virtual machine resource capacity policy, said policy containing required attributes of virtual machine resources of different types and different tiers reserved or allocated for a set of applications;

periodically collecting information from the storage environment components, the reservation repository, and the capacity policy repository and identifying a violation; and

displaying the violation on the display.

13. The system of claim 12, wherein a violation is selected from the group of:

a virtual machine resource that is allocated to an application but is not accessible to that application via an access path;

a virtual machine resource that is reserved by an application but not allocated to said application within a pre-selected interval of time;

a virtual machine resource that is allocated to an application but with a type or a tier not consistent with the requirements of the capacity policy repository;

an amount of resources reserved by an application or allocated to an application not consistent with the requirements of the capacity policy repository; and

an amount of virtual machine resources reserved or allocated to a set of storage environment components not consistent with the requirements of the capacity policy repository.

14. The system of claim 12, wherein the processor includes:

a timestamp processor for associating a timestamp with an event selected from the group of a virtual machine resource allocation event, a virtual machine resource reservation event, a virtual machine resource allocation state, and a virtual machine resource reservation state;

a data memory storing a list of past and present virtual machine resource allocation states, virtual machine resource reservation states, virtual machine resource allocation events, and virtual machine resource reservations events; and

a query processor for using the list to provide responses to queries about virtual machine resource allocation and virtual machine resource reservation at a given time.

15. The system of claim 12, wherein the processor includes:

a projection processor for computing projections of virtual machine allocation states and virtual machine reservation states and of virtual machine resources of specific types and tiers, wherein the computation involves extrapolating and trending from past virtual machine allocation states and past virtual machine reservation states, and using external input from authorized users; and

a query processor for using the computed projections to provide responses to queries from a designated user interacting with the user interface about virtual machine resource allocation and virtual machine resource reservation at a future time.

16. The system of claim 15, wherein the processor includes a utilization processor for estimating for a component type or tier, the level of storage capacity utilization.

17. The system of claim 16, wherein the level of storage capacity utilization is the percent of storage capacity utilized by the component.

18. The system of claim 16, wherein the utilization processor computes the level of utilization by computing a ratio of an amount of allocated and reserved virtual machine resources of a storage environment component, a virtual machine resource type, and a virtual machine resource tier, to the amount of allocated and reserved virtual machine resources corresponding to the storage environment component, the virtual machine resource type, or the virtual machine resource tier;

and providing violation notifications if the computed ratio is higher or lower than a pre-selected threshold in the virtual machine capacity policy repository.

19. The system of claim 17, wherein the processor includes a comparison processor for:

periodically computing current virtual machine allocation and reservations of different applications and different virtual machine resources types and tiers;

comparing the current virtual machine allocation and reservation with the computed projections; and

generating and displaying a projection violation notification on the display for use by a designated user, updating the computed projections, and sending the notification to a designated user.

20. A method for managing the capacity of virtual machine resources associated with applications in a storage network environment, comprising:

collecting information from sources in the network environment and deriving access paths defining end-to-end access relationships between an application on a virtual server and data objects associated with the virtual server in the network environment, wherein an access path includes a sequence of components configured to enable information flow between an application residing on a virtual machine and a data object on a respective storage device;

establishing rank classifications representative of the importance of a virtual machine resource in the network, and based on a level of storage service supported by the virtual machine;

computing for an access path associated with respective virtual machines the amount of its allocated and reserved virtual machine resources and the corresponding rank classifications;

collecting information about virtual machine resources from a virtual machine management platform and rank classifications reserved for a host application from a storage management platform;

accumulating for an application the amount of allocated and reserved virtual machine resources allocated to the application and classifying allocated and reserved virtual machine resources tiers and into allocated and reserved virtual machine resources on access paths and allocated and reserved virtual machine resources not on access paths;

constructing a virtual machine resource capacity policy, said policy containing required attributes of virtual machine resources of different types and different tiers reserved or allocated for a set of applications; and

periodically collecting information from the storage environment components, the reservation repository, and the capacity policy repository and identifying a violation.

21. The method of claim 20, wherein a violation is selected from the group of:

a virtual machine resource that is allocated to an application but is not accessible to that application via an access path;

a virtual machine resource that is reserved by an application but not allocated to said application within a pre-selected interval of time;

a virtual machine resource that is allocated to an application but with a type or a tier not consistent with the requirements of the capacity policy repository;

an amount of resources reserved by an application or allocated to an application not consistent with the requirements of the capacity policy repository; and

an amount of virtual machine resources reserved or allocated to a set of storage environment components not consistent with the requirements of the capacity policy repository.

22. The method of claim 20, further comprising:

associating a timestamp with a virtual machine resource allocation event and a respective virtual machine resource reservation event;

associating a timestamp with a virtual machine resource allocation state and updating the timestamp after a new virtual machine resource allocation event;

associating a timestamp with a virtual machine resource reservation state and updating the timestamp after a new virtual machine resource reservation event;

maintaining a list of past and present virtual machine resource allocation states, virtual machine resource reservation states, virtual machine resource allocation events, and virtual machine resource reservations events; and

using the list to provide responses to queries about virtual machine resource allocation and virtual machine resource reservation at a given time.

23. The method of claim 22, further comprising

computing projections of virtual machine allocation states and virtual machine reservation states and of virtual machine resources of specific types and tiers, wherein the computation involves extrapolating and trending from past virtual machine allocation states and past virtual machine reservation states, and using external input from a designated user;

and

using the computed projections to provide responses to queries about virtual machine resource allocation and virtual machine resource reservation at a future time.

24. The method of claim 23, further comprising estimating for a component type or tier, a level of storage capacity utilization.

25. The method of claim 24, wherein the level of utilization is the percent of storage capacity utilized by the component.

26. The method of claim 24, wherein the level of utilization is computed as a ratio of an amount of allocated and reserved virtual machine resources of a storage environment component, a virtual machine resource type, and a virtual machine resource tier, to the amount of allocated and reserved virtual machine resources corresponding to the storage environment component, the virtual machine resource type, or the virtual machine resource tier;

and providing violation notifications if the computed ratio is higher or lower than a pre-selected threshold in the virtual machine capacity policy repository.

27. The method of claim 26, further comprising:

periodically computing current virtual machine allocation and reservations of different applications and different virtual machine resources types and tiers;

comparing the current virtual machine allocation and reservation with the computed projections;

generating a projection violation notification, updating the computed projections, and sending the notification to a designated user.