Systems and methods for path-based tier-aware dynamic capacity management in storage network environments

• US 8,332,860 B1
• Filed: 12/31/2007
• Issued: 12/11/2012
• Est. Priority Date: 12/30/2006
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Yahalom, Raphael, Levy, Assaf
• Primary Examiner(s): An, Meng
• Assistant Examiner(s): Ghaffari, Abu Z
• Application Number:
  US12/006,125
• Time in Patent Office: 1,807 Days
• Field of Search: 718/104, 709/226
• US Class Current: 718/104
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A dynamic approach for managing capacity in a storage environments accounting for various resource types, tiers, access paths, relationships among different storage environment components, as well as managing capacity in terms of resource planning and consumption is provided. Resource planning and consumption processes are used to allocate and reserve resources of different types and different tiers to each access path in the network. Capacity information for different resources types and tiers is automatically aggregated from different network components at different stages of a resource management cycle. The aggregated information may be used to estimate future resource needs or determine deviations between projected and actual status from which adjustments may be made to better predict future capacity needs.

Claims

1. A computerized process for periodically analyzing a capacity level of component device resources associated with applications in a storage network environment, comprising:

collecting information from the component device resources in the storage network environment;

deriving one or more access paths in the storage network environment associated with each application of a plurality of applications, based on the collected information, wherein each access path comprises an end-to-end logical path between the associated application and a component device resource used by the associated application;

determining a tier level for each of the component device resources in the storage network environment representative of a level of storage service supported by the resource;

computing, for each access path associated with an application, a total amount of resources associated with the access path that have been allocated to the access path and the tier levels of the allocated resources;

collecting information about the total amount of resources that have been reserved for each application and the tier levels of the reserved resources, wherein the reserved resources cannot be used by the application unless allocated, and cannot be allocated or reserved to another application;

classifying the allocated and reserved resources for each application into tiers, into allocated and reserved resources on access paths associated with the application, and into allocated and reserved resources not on access paths associated with the application, whereby the allocated and reserved resources not on the access paths associated with the application are inaccessible by the application.

2. The computerized process of claim 1, wherein a reservation repository contains information about the allocated and reserved resources for each application in the storage network environment at a current time.

3. The computerized process of claim 1, wherein deriving the one or more access paths comprises mapping network component configuration and connectivity information to nodes and vertices of an abstract graph and determining whether information can flow between end nodes in the graph.

4. The computerized process of claim 1, wherein determining the tier levels for each resource comprises setting tier assignment rules, deriving a type, an attribute value, and an annotation value of the resource, and using rules to determine the tier level of the resource based on the type, the attribute value and the annotation value.

5. The computerized process of claim 1, wherein the tier level for each resource is a ranking of normalized value ranges characterizing the storage service levels, wherein the normalized value ranges include one of ordinal and numeric values.

6. The computerized process of claim 1, further comprising continuously monitoring the network environment and a reservation repository;

detecting state change events and obtaining updated state information;

using the updated state information to compute for each application the accumulated total amount of allocated and reserved resources allocated to the application and their tier levels; and

using the updated state information to analyze the allocated and reserved resources allocated to or reserved for each application and determine whether the allocated and reserved resources are currently accessible by the application via an access path.

7. The computerized process of claim 1, further comprising periodically generating summary reports containing:

current total amount of allocated and reserved resources;

a ratio of the total amount of allocated and reserved resources to an amount of total available component device resources in the corresponding component, or to an amount of allocated and reserved resources at other components of the same type and tier; and

a cost of the allocated or reserved resources.

8. The computerized process of claim 7, further comprising providing information in the reports on component power-consumption, component heat generation, and component physical space consumption.

9. A capacity manager for managing a capacity level of component device resources associated with applications in a storage network environment, comprising a processor for:

collecting information from the component device resources in the storage network environment;

deriving one or more access paths in the storage network environment associated with each application of a plurality of applications, based on the collected information, wherein each access path comprises an end-to-end logical path between the associated application and a component device resource used by the associated application;

determining a tier level for each of the component device resources in the storage network environment representative of a level of storage service supported by the resource;

computing, for each access path associated with an application, a total amount of resources associated with the access path that have been allocated and reserved resources to the access path and the tier levels of the allocated resources;

collecting information about the total amount of resources that have been reserved for each application and the tier levels of the reserved resources, wherein the reserved resources cannot be used by the application unless allocated, and cannot be allocated or reserved to another application;

classifying allocated and reserved resources for each application tiers, into allocated and reserved resources on access paths associated with the application, and into allocated and reserved resources not on access paths associated with the application, whereby the allocated and reserved resources not on the access paths associated with the application are inaccessible by the application;

constructing a resource capacity policy repository, said policy repository containing required attributes of resources of different types and different tiers reserved or allocated for the plurality of applications;

periodically collecting information from the component device resources in the storage network environment, a reservation repository, and the resource capacity policy repository to identify a violation, wherein the reservation repository contains information about the allocated and reserved resources for each of the applications in the storage network environment at a current time; and

sending notification messages to designated recipients with the identified violation.

10. The capacity manager of claim 9, wherein a violation includes at least one of:

a resource that is allocated to an application but is not accessible to that application via an access path;

a resource that is reserved by an application but not allocated to said application within a pre-selected interval of time;

a resource that is allocated to an application but with a type or a tier not consistent with the requirements of the resource capacity policy repository;

a total amount of resources reserved by an application or allocated to an application not consistent with the requirements of the resource capacity policy repository; and

a total amount of resources reserved or allocated to a set of storage environment components not consistent with the requirements of the resource capacity policy repository.

11. The capacity manager of claim 9, further comprising:

associating a timestamp with each resource allocation event and each resource reservation event;

associating a timestamp with each resource allocation state and updating the timestamp after a new resource allocation event;

associating a timestamp with each resource reservation state and updating the timestamp after a new resource reservation event;

maintaining a list of past and present resource allocation states, resource reservation states, resource allocation events, and resource reservations events; and

using the list to provide responses to queries about resource allocation and resource reservation at a given time.

12. The capacity manager of claim 11, further comprising:

computing projections of allocation states and reservation states and of resources of specific types and tiers, wherein the computation involves extrapolating and trending from past allocation states and past reservation states, and using external input from authorized users; and

using the computed projections to provide responses to queries about resource allocation and resource reservation at a future time.

13. The capacity manager of claim 12, further comprising estimating for each component, type, or tier, a level of utilization.

14. The capacity manager of claim 13, wherein the level of utilization is computed by computing a ratio of an amount of allocated and reserved resources of a storage environment component, a resource type, and a resource tier, to the total amount of allocated and reserved resources corresponding to the storage environment component, the resource type, or the resource tier;

and providing violation notifications if the computed ratio is higher or lower than a pre-selected threshold in the capacity policy repository.

15. The capacity manager of claim 14, further comprising:

periodically computing current allocation and reservations of different applications and different resources types and tiers;

comparing the current allocation and reservation with the computed projections;

generating a projection violation notification, updating the computed projections, and sending the notification to a designated recipient.

16. A computerized process for periodically analyzing a capacity level of component device resources associated with applications in a storage network environment, comprising:

deriving one or more access paths in the storage network environment associated with each application, wherein each access path comprises an end-to-end logical path between the application and one of the component device resources used by the application;

determining, for each derived access path, the component device resources associated with the access path that have been allocated to the access path;

determining, for each of the applications, the component device resources that have been reserved for the application, wherein the reserved resources cannot be used by the application unless allocated, and cannot be allocated or reserved to another application;

determining a tier level for each of the component device resources representative of a level of storage service supported by each of the component device resources;

classifying the allocated and reserved resources for each application into allocated and reserved resources on the access paths associated with the application, and into allocated and reserved resources not on access paths associated with the application, whereby the allocated and reserved resources not on the access paths associated with the application are inaccessible by the application; and

classifying further the allocated and reserved component device resources for each of the applications into tiers.

17. A capacity manager for managing a capacity level of component device resources associated with applications in a storage network environment, comprising a processor configured to:

derive one or more access paths in the storage network environment associated with each application, wherein each access path comprises an end-to-end logical path between the application and one of the component device resources used by the application;

determine, for each derived access path, the component device resources associated with the access path that have been allocated to the access path;

determine, for each of the applications, the component device resources that have been reserved for the application, wherein the reserved resources cannot be used by the application unless allocated, and cannot be allocated or reserved to another application;

determine a tier level for each of the component device resources representative of a level of storage service supported by each of the component device resource;

classify the allocated and reserved resources for each application into allocated and reserved resources on the access paths associated with the application, and into allocated and reserved resources not on access paths associated with the application, whereby the allocated and reserved resources not on the access paths associated with the application are inaccessible by the application; and

classify further the allocated and reserved component device resources for each of the applications into tiers.

18. A computerized process for periodically analyzing a capacity level of component device resources associated with applications in a storage network environment, comprising:

deriving one or more access paths in the storage network environment associated with each application, wherein each access path comprises an end-to-end logical path between the application and one of the component device resources used by the application;

determining, for each derived access path, the component device resources associated with the access path that have been allocated to the access path;

determining, for each of the applications, the component device resources that have been reserved for the application, wherein the reserved resources cannot be used by the application unless allocated, and cannot be allocated or reserved to another application;

classifying the allocated and reserved resources for each application into allocated and reserved resources on the access paths associated with the application, and into allocated and reserved resources not on access paths associated with the application, whereby the allocated and reserved resources not on the access paths associated with the application are inaccessible by the application;

constructing a resource capacity policy repository, said policy repository containing required attributes of resources of different types and different tiers reserved or allocated for each application;

periodically collecting information from the component device resources, a reservation repository, and the resource capacity policy repository to identify a violation, wherein the reservation repository contains information about the allocated and reserved resources for each application at a current time; and

sending notification messages to designated recipients with the identified violation.

19. A capacity manager for managing a capacity level of component device resources associated with applications in a storage network environment, comprising a processor configured to:

derive one or more access paths in the storage network environment associated with each application, wherein each access path comprises an end-to-end logical path between the application and one of the component device resources used by the application;

determine, for each derived access path, the component device resources associated with the access path that have been allocated to the access path;

determine, for each of the applications, the component device resources that have been reserved for the application, wherein the reserved resources cannot be used by the application unless allocated, and cannot be allocated or reserved to another application;

classify the allocated and reserved resources for each application into allocated and reserved resources on the access paths associated with the application, and into allocated and reserved resources not on access paths associated with the application, whereby the allocated and reserved resources not on the access paths associated with the application are inaccessible by the application;

construct a resource capacity policy repository, said policy repository containing required attributes of resources of different types and different tiers reserved or allocated for each application;

periodically collect information from the component device resources, a reservation repository, and the resource capacity policy repository to identify a violation, wherein the reservation repository contains information about the allocated and reserved resources for each application at a current time; and

send notification messages to designated recipients with the identified violation.