Graphical representation of classification of workloads

• US 9,276,821 B2
• Filed: 07/18/2013
• Issued: 03/01/2016
• Est. Priority Date: 07/18/2013
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Basak, Jayanta
• Primary Examiner(s): To, Jennifer
• Assistant Examiner(s): LI, LIANG Y
• Application Number:
  US13/945,797
• Publication Number:
  US 20150026582A1
• Time in Patent Office: 957 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data

Abstract

Technology is disclosed for graphically representing classification of workloads in a storage system. Workload classification is graphically represented to the user by mapping workload signatures of the workloads to a grid. When the workload signatures are mapped to the grid, a number of clusters are formed in the grid. Each of the clusters represents workloads of a particular category. Mapping the workload signature to the grid includes mapping a high-dimensionality workload signature vector to a low-dimensionality grid.

Claims

1. A method, comprising:

receiving, at a processing device in a network storage system, workload signatures corresponding to workloads in the network storage system, each of the workload signatures being a vector having a first dimensionality and each workload signature classifies a runtime workload based on data gathered by sampling operations associated with the workload and comprises of a plurality of parameters associated with the workload, a response time by a network controller of the network storage system to respond to the workload and at least one parameter that is significant for predicting the response time, and the at least one parameter is determined using statistical analysis of a plurality of sampled values to determine a correlation of the at least one parameter to the response time;

mapping, by the processing device, each of the workload signatures to a grid of a second dimensionality, where the grid is trained using a sample data set with predefined number of sample workload signatures across known workload categories, and the second dimensionality being lower than the first dimensionality, the mapping including;

identifying, based on the vector of a particular workload signature, a particular cell of the grid that matches with the vector, the identifying including determining whether the particular cell matches with the vector based on a weight vector associated with the particular cell, and

assigning the particular workload signature to the particular cell; and

generating, by the processing device, a graphical representation of classification of the workloads using the grid, the grid containing clusters, the clusters formed by mapping the workload signatures, each of the clusters including a group of cells of the grid and identifying a category of the workloads whose workload signatures are assigned to the group of cells of the corresponding cluster.

2. The method of claim 1, wherein mapping the workload signatures includes mapping the workload signatures with similar vectors to adjacent cells in a common region of the grid, the region forming one of the clusters.

3. The method of claim 2 further comprising:

receiving, at regular intervals of time, a new set of workload signatures of the workloads; and

updating, at regular intervals of time, the graphical representation to indicate the classification of the workloads at a given interval of time.

4. The method of claim 2, further comprising:

generating, by the processing device, a report including analytics of the workloads in the network storage system.

5. The method of claim 1, wherein mapping the workload signature to the grid of a second dimensionality includes:

calibrating, by the processing device and using a topographic mapping technique, the grid using a sample data set, the sample data set including multiple sample workload signatures of multiple sample workloads, the sample workloads categorized into predefined categories, the sample data set including at least one sample workload signature in each of the categories, the calibrating including for each of the categories,

assigning the sample workload signatures to cells in the grid, the cells forming a first cluster in the grid, and

identifying the first cluster as representing the corresponding category of the sample workload.

6. The method of claim 5, wherein calibrating the grid includes calibrating the grid before mapping the workload signatures to the grid.

7. The method of claim 1, wherein receiving the workload signatures includes:

sampling, by the processing device and for a specified duration, values for parameters of a workload of the workloads,

determining, based at least in part on the sampled values, a subset of the parameters for predicting a response time of a network storage controller executing the workload, the response time being the time required for the network storage controller to respond to a service request from the particular workload, and

generating, by the processing device, the vector of the workload signature as a function of the values of the parameters, the response time, and the subset of the parameters.

8. The method of claim 7, wherein the parameters include at least one of

a percentage of read operations of a total operation performed by the corresponding workload;

a percentage of write operations of the total operations performed by the corresponding workload;

a percentage of the total operations that are neither read nor write performed by the corresponding workload;

an average size of data read by the corresponding workload;

an average size of data written by the corresponding workload;

a percentage of random read operations of a total real operation performed by the corresponding workload;

or

a percentage of random write operations of a total write operation performed by the corresponding workload.

9. The method of claim 8, wherein at least some of the parameters of the workload signatures mapped to the same cluster have similar values.

10. The method of claim 7, wherein the first dimensionality of the vector is of size sixty four.

11. The method of claim 10, wherein the parameters are represented in the vector as sixty four bits.

12. The method of claim 1, wherein the grid of the second dimensionality is a two-dimensional grid of a predefined size.

13. The method of claim 1, wherein each of the workloads is executing on at least one of a disk of a network storage controller, an aggregate of the network storage controller, or a volume of the aggregate.

14. The method of claim 1 further comprising:

receiving a new set of workload signatures of a new set of workloads;

mapping, using a topography mapping technique, the new set of workload signatures to the grid;

determining if a new cluster is formed by mapping the new set of workload signatures; and

responsive to a determination that the new cluster is formed in the grid, identifying the new cluster as representing a new category of the new set of workloads, the new category being different from categories of previously mapped workload signatures.

15. A non-transitory, machine readable medium having stored thereon instructions comprising machine executable code which when executed by a machine, causes the machine to:

receive, at a processing device in a network storage system, workload signatures corresponding to workloads in the network storage system, each of the workload signatures being a vector having a first dimensionality and each workload signature classifies a runtime workload based on data gathered by sampling operations associated with the workload and comprises of a plurality of parameters associated with the workload, a response time by a network controller of the network storage system to respond to the workload and at least one parameter that is significant for predicting the response time, and the at least one parameter is determined using statistical analysis of a plurality of sampled values to determine a correlation of the at least one parameter to the response time;

map, by the processing device, each of the workload signatures to a grid of a second dimensionality, where the grid is trained using a sample data set with predefined number of sample workload signatures across known workload categories, and the second dimensionality being lower than the first dimensionality, the mapping includes;

identify, based on the vector of a particular workload signature, a particular cell of the grid that matches with the vector, the identifying including determining whether the particular cell matches with the vector based on a weight vector associated with the particular cell, and

assign the particular workload signature to the particular cell; and

generate, by the processing device, a graphical representation of classification of the workloads using the grid, the grid containing clusters, the clusters formed by mapping the workload signatures, each of the clusters including a group of cells of the grid and identifying a category of the workloads whose workload signatures are assigned to the group of cells of the corresponding cluster.

16. The non-transitory storage medium of claim 15, wherein mapping the workload signature to the grid of a second dimensionality includes:

calibrating, by the processing device and using a topographic mapping technique, the grid using a sample data set, the sample data set including multiple sample workload signatures of multiple sample workloads, the sample workloads categorized into predefined categories, the sample data set including at least one sample workload signature in each of the categories, the calibrating including for each of the categories,

assigning the sample workload signatures to cells in the grid, the cells forming a first cluster in the grid, and

identifying the first cluster as representing the corresponding category of the sample workload.

17. The non-transitory, storage medium of claim 15, wherein receiving the workload signatures includes:

sampling, by the processing device and for a specified duration, values for parameters of a workload of the workloads,

determining, based at least in part on the sampled values, a subset of the parameters for predicting a response time of a network storage controller executing the workload, the response time being the time required for the network storage controller to respond to a service request from the particular workload, and

generating, by the processing device, the vector of the workload signature as a function of the values of the parameters, the response time, and the subset of the parameters.

18. A system, comprising:

a memory containing machine readable medium comprising machine executable code having stored thereon instructions; and

a processing device in a networked storage system coupled to the memory, the processor device configured to execute the machine executable code to;

receive workload signatures corresponding to workloads in the network storage system, each of the workload signatures being a vector having a first dimensionality and each workload signature classifies a runtime workload based on data gathered by sampling operations associated with the workload and comprises of a plurality of parameters associated with the workload, a response time by a network controller of the network storage system to respond to the workload and at least one parameter that is significant for predicting the response time, and the at least one parameter is determined using statistical analysis of a plurality of sampled values to determine a correlation of the at least one parameter to the response time;

map each of the workload signatures to a grid of a second dimensionality, where the grid is trained using a sample data set with predefined number of sample workload signatures across known workload categories, and the second dimensionality being lower than the first dimensionality, the mapping includes;

identify, based on the vector of a particular workload signature, a particular cell of the grid that matches with the vector, the identifying including determining whether the particular cell matches with the vector based on a weight vector associated with the particular cell, and

assign the particular workload signature to the particular cell; and

generate a graphical representation of classification of the workloads using the grid, the grid containing clusters, the clusters formed by mapping the workload signatures, each of the clusters including a group of cells of the grid and identifying a category of the workloads whose workload signatures are assigned to the group of cells of the corresponding cluster.

19. The system of claim 18, wherein mapping the workload signature to the grid of a second dimensionality includes:

calibrating, by the processing device and using a topographic mapping technique, the grid using a sample data set, the sample data set including multiple sample workload signatures of multiple sample workloads, the sample workloads categorized into predefined categories, the sample data set including at least one sample workload signature in each of the categories, the calibrating including for each of the categories,

assigning the sample workload signatures to cells in the grid, the cells forming a first cluster in the grid, and

identifying the first cluster as representing the corresponding category of the sample workload.

20. The system of claim 19, wherein receiving the workload signatures includes:

sampling, by the processing device and for a specified duration, values for parameters of a workload of the workloads,

determining, based at least in part on the sampled values, a subset of the parameters for predicting a response time of a network storage controller executing the workload, the response time being the time required for the network storage controller to respond to a service request from the particular workload, and

generating, by the processing device, the vector of the workload signature as a function of the values of the parameters, the response time, and the subset of the parameters.