Method, Apparatus and Computer Program Product for Intelligent Workload Control of Distributed Storage

• US 20090157378A1
• Filed: 12/17/2007
• Published: 06/18/2009
• Est. Priority Date: 12/17/2007
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Nokia Corporation
• Inventors: Tyrkko, Olli Teppo Kalevi, Boldyrev, Sergey, Lappetelainen, Antti Tuomas
• Granted Patent: US 8,005,654 B2
• Time in Patent Office: Days
• US Class Current: 703/22
• CPC Class Codes: G06F 11/008 Reliability or availability analysis

Abstract

A method, apparatus and computer program product are provided for generating a multiple-input, multiple output (MIMO) workload model of a distributed storage environment on a plurality of predictive controller devices. The MIMO models can then be utilized by the predictive controller to manage storage resources on a distributed storage system.

Claims

1. A method comprising:

generating a multiple-input, multiple-output (MIMO) workload model of a distributed storage environment on at least one predictive controller; and

utilizing the MIMO workload model on at least one predictive controller to manage storage resources on a distributed storage system.

2. The method of claim 1 wherein generating a MIMO workload model of a distributed storage environment on at least one predictive controller comprises:

converting a plurality of requests per some predefined period of time into a moving average;

performing a frequency domain transformation of the moving average;

identifying a workload model through spectral analysis of the moving average;

extracting spectral components from the transformed moving average;

predicting break points for reliable and non-reliable nodes; and

performing an efficiency estimation to refine the workload model using, in part, cost functions and cost weighting factors based on at least one of power consumption, performance, and reliability.

3. The method of claim 2, wherein the method is performed separately for read and write requests.

4. The method of claim 2 wherein the element of generating a MIMO workload model of a distributed storage environment on a plurality of predictive controllers further comprises:

performing an error level estimation and determining an associated threshold, and tracking an error level against the threshold.

5. The method of claim 1, further comprising correlating the MIMO workload models on each of the at least one predictive controllers into a system MIMO workload model.

6. The method of claim 1, further comprising dispersing data using the at least one predictive controller wherein the at least one predictive controller uses an information dispersion algorithm for data dispersion, and aggregating data using the at least one predictive controller wherein the at least one predictive controller uses an information dispersion algorithm for data aggregating.

7. The method of claim 6, wherein data is dispersed according to a skewed dispersion scheme and data is aggregated according to a skewed dispersion scheme.

8. The method of claim 6, wherein data is dispersed according to a non-uniform dispersion scheme and data is aggregated according to a non-uniform dispersion scheme.

9. The method of claim 1, wherein each predictive controller in the plurality of predictive controllers comprises a multi-layered cache.

10. An apparatus comprising:

a feed forward workload estimator configured to receive a plurality of requests per some predefined period of time on a distributed storage system and generate a multiple-input, multiple-output (MIMO) workload model; and

a data dispersing element configured to disperse and aggregate data with an information dispersing algorithm.

11. The apparatus of claim 10, further comprising a multi-layered cache.

12. The apparatus of claim 10, wherein the feed forward workload estimator being configured to generate a MIMO workload, comprises being configured to

convert the plurality of requests per some predefined period of time into a moving average;

perform a frequency domain transformation of the moving average;

identify a workload model through spectral analysis of the moving average;

extract spectral components from the transformed moving average;

predict break points for reliable and non-reliable nodes; and

perform an efficiency estimation to refine the workload model using, in part, cost functions and cost weighting factors based on power consumption, performance, and reliability.

13. The apparatus of claim 10, wherein the feed forward workload estimator is further configured to perform an error level estimation and determine an associated threshold, and track an error level against the threshold.

14. The apparatus of claim 10, wherein the data dispersing element is configured to disperse and aggregate data according to a skewed dispersion scheme.

15. The apparatus of claim 10, wherein the data dispersing element is configured to disperse and aggregate data according to a non-uniform dispersion scheme.

16. An apparatus comprising:

means for generating a multiple-input, multiple-output (MIMO) workload model of a distributed storage environment;

means for utilizing the MIMO workload model to manage storage resources on a distributed storage system.

17. The apparatus of claim 16, further comprising a means for correlating a plurality of MIMO workload models into a system MIMO workload model.

18. A computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion configured to generate a multiple-input, multiple-output (MIMO) workload model of a distributed storage environment; and

a second executable portion configured to manage storage resources on a distributed storage system by utilizing the MIMO workload model.

19. The computer program product of claim 18, wherein the first executable portion configured to generate a MIMO workload model of a distributed storage environment is further configured to:

convert a plurality of requests per some predefined period of time into a moving average;

perform a frequency domain transformation of the moving average;

identify a workload model through spectral analysis of the moving average;

extract spectral components from the transformed moving average;

predict break points for reliable and non-reliable nodes; and

perform an efficiency estimation to refine the workload model using, in part, cost functions and weighting factors based on power consumption, performance, and reliability.

20. The computer program product of claim 18, further comprising a third executable portion configured to correlate multiple MIMO workload models into a system MIMO model.

21. The computer program product of claim 19, wherein the first executable portion configured to generate a MIMO workload model of a distributed storage environment is further configured to:

perform an error level estimation and determine an associated threshold; and

track an error level against the threshold.

22. The computer program product of claim 18, wherein the second executable portion is further configured to disperse data using an information dispersion algorithm and aggregate data using an information dispersion algorithm.

23. The computer program product of claim 22, wherein the second executable portion is further configured to disperse data according to a skewed dispersion scheme and aggregate data according to a skewed dispersion scheme.

24. The computer program product of claim 22, wherein the second executable portion is further configured to disperse data according to a non-uniform dispersion scheme and aggregate data according to a non-uniform dispersion scheme.

25. The computer program product of claim 18, wherein the second executable portion is further configured to implement a multi-layered cache.