Methods and systems using observation based techniques for determining performance capacity of a resource of a networked storage environment

• US 9,594,515 B2
• Filed: 07/22/2015
• Issued: 03/14/2017
• Est. Priority Date: 07/22/2015
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Hrischuk, Curtis, Dimnaku, Alma, Fairbanks, Leon, Boswell, Steven B.
• Primary Examiner(s): NGUYEN, HIEP T
• Application Number:
  US14/805,829
• Publication Number:
  US 20170024144A1
• Time in Patent Office: 601 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Methods and systems for managing resources in a networked storage environment are provided. One method includes generating a relationship between latency and utilization of a resource in a networked storage environment using observation based, current and historical latency and utilization data, where latency is an indicator of delay at the resource for processing any request and utilization of the resource is an indicator of an extent the resource is being used at any given time; and selecting an optimal point for the generated relationship between latency and utilization, where the optimal point is an indicator of resource utilization beyond which throughput gains for a workload is smaller than increase in latency.

Claims

1. A machine implemented method, comprising:

generating a relationship between latency and utilization of a resource from among a plurality of resources in a networked storage environment using a first observation based technique that uses collected current and historical latency and utilization data associated with the resource, where latency is an indicator of delay at the resource for processing any request and utilization of the resource is an indicator of an extent the resource is being used at any given time;

generating the relationship between latency and utilization using a second model based technique that uses inter-arrival-times and service times for the resource for processing a plurality of requests, where inter-arrival times indicate arrival times for the plurality of requests at the resource and service times indicate durations for servicing the plurality of request requests;

selecting an optimal point based on a confidence level for the generated relationship between latency and utilization using the first observation based technique and the second model based technique, where the optimal point is an indicator of resource utilization beyond which throughput gains for a workload is smaller than increase in latency;

identifying a plurality of operational points associated with the resource, where the operational points vary based on a service level objective, whether the resource is part of a high availability node pair within the networked storage system and when workloads by the networked storage system are throttled for processing; and

determining an available performance capacity for the resource based on the optimal point and the plurality of operational points.

2. The machine implemented method of claim 1, wherein the resource is a processor for a networking module that provides networking functionality in the networked storage environment and a processor for a storage module that interfaces with a storage device for storing data.

3. The machine implemented method of claim 2, wherein the resource is an aggregate that includes the storage device.

4. The machine implemented method of claim 1, wherein historical latency data is used to interpolate any missing data from the current latency data.

5. The machine implemented method of claim 1, wherein when an incomplete curve between latency and utilization is generated based on current and historical latency and utilization data, then the incomplete curve is completed by extrapolation.

6. The machine implemented method of claim 1, wherein the available performance capacity of the resource varies based on the plurality of operational points.

7. The machine implemented method of claim 1, wherein to generate the latency and utilization relationship, a service time for the plurality of requests is used to search for historical latency and utilization data for augmenting the current latency and utilization data.

8. A non-transitory, machine readable storage medium having stored thereon instructions for performing a method, comprising machine executable code which when executed by at least one machine, causes the machine to:

generate a relationship between latency and utilization of a resource from among a plurality of resources in a networked storage environment using a first observation based technique that uses collected current and historical latency and utilization data associated with the resource, where latency is an indicator of delay at the resource for processing any request and utilization of the resource is an indicator of an extent the resource is being used at any given time;

generate the relationship between latency and utilization using a second model based technique that uses inter-arrival-times and service times for the resource for processing a plurality of requests, where inter-arrival times indicate arrival times for the plurality of requests at the resource and service times indicate durations for servicing the plurality of request;

select an optimal point based on a confidence level for the generated relationship between latency and utilization using the first observation based technique and the second model based technique, where the optimal point is an indicator of resource utilization beyond which throughput gains for a workload is smaller than increase in latency;

identify a plurality of operational points associated with the resource, where the operational points vary based on a service level objective, whether the resource is part of a high availability node pair within the networked storage system and when workloads by the networked storage system are throttled for processing; and

determine an available performance capacity for the resource based on the optimal point and the plurality of operational points.

9. The storage medium of claim 8, wherein the resource is a processor for a networking module that provides networking functionality in the networked storage environment and a processor for a storage module that interfaces with a storage device for storing data.

10. The storage medium of claim 9, wherein the resource is an aggregate that includes the storage device.

11. The storage medium of claim 8, wherein historical latency data is used to interpolate any missing data for current latency data.

12. The storage medium of claim 8, wherein when an incomplete curve between latency and utilization is generated based on current and historical latency and utilization data, then the incomplete curve is completed by extrapolation.

13. The storage medium of claim 8, wherein the available performance capacity of the resource varies based on the plurality of operational points.

14. The storage medium of claim 8, wherein to generate the latency and utilization relationship, a service time for the plurality of requests is used to search for historical latency and utilization data for augmenting the current latency and utilization data.

15. A system comprising:

a memory containing machine readable medium comprising machine executable code having stored thereon instructions; and

a processor module coupled to the memory, the processor module configured to execute the machine executable code to;

generate a relationship between latency and utilization of a resource from among a plurality of resources in a networked storage environment using a first observation based technique that uses collected current and historical latency and utilization data associated with the resource, where latency is an indicator of delay at the resource for processing any request and utilization of the resource is an indicator of an extent the resource is being used at any given time;

generate the relationship between latency and utilization using a second model based technique that uses inter-arrival-times and service times for the resource for processing a plurality of requests, where inter-arrival times indicate arrival times for the plurality of requests at the resource and service times indicate durations for servicing the plurality of request requests;

select an optimal point based on a confidence level for the generated relationship between latency and utilization using the first observation based technique and the second model based technique, where the optimal point is an indicator of resource utilization beyond which throughput gains for a workload is smaller than increase in latency;

identify a plurality of operational points associated with the resource, where the operational points vary based on a service level objective, whether the resource is part of a high availability node pair within the networked storage system and when workloads by the networked storage system are throttled for processing; and

determine an available performance capacity for the resource based on the optimal point and the plurality of operational points.

16. The system of claim 15, wherein the resource is a processor for a networking module that provides networking functionality in the networked storage environment and a processor for a storage module that interfaces with a storage device for storing data.

17. The system of claim 16, wherein the resource is an aggregate that includes the storage device.

18. The system of claim 15, wherein historical latency data is used to interpolate any missing data for current latency data.

19. The system of claim 15, wherein when an incomplete curve between latency and utilization is generated based on current and historical latency and utilization data, then the incomplete curve is completed by extrapolation.

20. The system of claim 15, wherein the available performance capacity of the resource varies based on the plurality of operational points.

21. The system of claim 15, wherein to generate the latency and utilization relationship, a service time for the plurality of requests is used to search for historical latency and utilization data for augmenting the current latency and utilization data.