Storage application performance matching

• US 8,380,947 B2
• Filed: 04/27/2012
• Issued: 02/19/2013
• Est. Priority Date: 02/05/2010
• Status: Expired due to Fees

• Current Assignee: AirBNB Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Chiu, Lawrence Y., Dickey, Clement L., Hsu, Yu-Cheng, Hyde, Joseph S. II, Muench, Paul H., Song, Cheng-Chung
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): Faal, Baboucarr
• Application Number:
  US13/458,843
• Publication Number:
  US 20120215949A1
• Time in Patent Office: 298 Days
• Field of Search: None
• US Class Current: 711/156
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

Input/output (I/O) activity in the multiple tier storage system is monitored to collect statistical information. The statistical information is recurrently transformed into an exponential moving average (EMA) of the I/O activity having a predefined smoothing factor. Data portions in the multiple tier storage system are sorted into buckets of varying temperatures corresponding to the EMA. At least one data migration plan is recurrently generated for matching the sorted data portions to at least one of an available plurality of storage device classes. One data portion sorted into a higher temperature bucket is matched with a higher performance storage device class of the available plurality of storage device classes than another data portion sorted into a lower temperature bucket.

Claims

1. A method for matching storage application performance in a multiple tier storage system by a processor device, comprising:

monitoring input/output (I/O) activity in the multiple tier storage system to collect statistical information;

recurrently transforming the statistical information of the I/O activity using an exponential moving average (EMA) having a predefined smoothing factor to obtain an EMA value;

sorting data portions in the multiple tier storage system into buckets of varying temperatures corresponding to the EMA value;

recurrently generating at least one data migration plan for matching the sorted data portions to at least one of an available plurality of storage device classes, wherein one data portion sorted into a higher temperature bucket having higher EMA values is matched with a higher performance storage device class of the available plurality of storage device classes than another data portion sorted into a lower temperature bucket having lower EMA values, wherein the plurality of storage device classes includes a fastest tier 0 and a number of slower tiers n;

identifying a hot candidate in one or more of the number of slower tiers n;

determining whether the hot candidate has an up trend; and

promoting the hot candidate to a faster tier.

2. The method of claim 1, wherein the sorting data portions includes sorting extents of the multiple tier storage system into the buckets of varying temperatures corresponding to the EMA value.

3. The method of claim 1, wherein the monitoring I/O activity in the multiple tier storage system is repeated for each of a plurality of predefined smoothing factors having decreasing values to generate at least a shorter and a longer EMA of the I/O activity.

4. The method of claim 3, further including performing one of:

pursuant to detecting that the longer EMA of the I/O activity is smaller than the shorter EMA of the I/O activity, adjusting the at least one data migration plan to consider a current up trend change of the I/O activity, and

pursuant to detecting that the longer EMA is larger than the shorter EMA of the I/O activity, adjusting the at least one data migration plan to consider current down trend change of the I/O activity.

5. The method of claim 1, wherein the generating at least one data migration plan includes performing at least one of:

promoting the one data portion sorted into the higher temperature bucket from one or more storage devices in a hard disk drive (HDD) class to one or more storage devices in a solid state drive (SSD) class,

demoting the another data portion sorted into the lower temperature bucket from the one or more storage devices in the SSD class to the one or more devices in the HDD class, and

swapping the another data portion sorted into the lower temperature bucket in the one or more storage devices in the SSD class with the one data portion sorted into the higher temperature bucket in the one or more storage devices in the HDD class.

6. The method of claim 1, further including allocating the at least one of the available plurality of storage device classes according to the generated at least one data migration plan, wherein pursuant to the allocating the at least one of the available plurality of storage device classes, a data migration bandwidth is throttled so as not to disrupt overall performance of the multiple tier storage system.

7. The method of claim 1, wherein the monitoring the I/O activity in the multiple tier storage to collect the statistical information includes collecting at least one of:

an I/O read count,

an I/O write count,

a cumulative latency,

a cache miss count,

a total data transferred, and

an average I/O size.

8. The method of claim 1, wherein the promoting includes generating a promoting data migration plan for the hot candidate.

9. The method of claim 8, wherein the generating includes determining whether a projected performance gain of the hot candidate resulting on the fastest tier 0 will be larger than a migration cost, wherein the projected performance gain is equal to a current performance minus a projected performance and the migration cost is calculated a priori in terms of I/O count and total latency.

10. The method of claim 9, wherein generating includes:

generating the migration plan, if the projected performance gain is a net gain; and

generating a swapping data migration plan, if the projected performance is not a net gain.

11. A method for matching storage application performance in a multiple tier storage system by a processor device, comprising:

monitoring input/output (I/O) activity in the multiple tier storage system to collect statistical information;

recurrently transforming the statistical information of the I/O activity using an exponential moving average (EMA) having a predefined smoothing factor to obtain an EMA value;

sorting data portions in the multiple tier storage system into buckets of varying temperatures corresponding to the EMA value;

recurrently generating at least one data migration plan for matching the sorted data portions to at least one of an available plurality of storage device classes, wherein one data portion sorted into a higher temperature bucket having higher EMA values is matched with a higher performance storage device class of the available plurality of storage device classes than another data portion sorted into a lower temperature bucket having lower EMA values, wherein the plurality of storage device classes includes a fastest tier 0 and a number of slower tiers n;

identifying a cold candidate in the fastest tier 0;

determining whether the cold candidate has a down trend; and

demoting the cold candidate from the faster tier 0 to one of the number of slower tiers n.

12. The method of claim 11, wherein identifying includes identifying the cold candidate from a temperature-sorted moving average list in fastest tier 0, wherein a tail of the temperature-sorted moving average list represents coldest data within the fastest tier 0.

13. The method of claim 11, wherein the monitoring I/O activity in the multiple tier storage system is repeated for each of a plurality of predefined smoothing factors having decreasing values to generate at least a shorter and a longer EMA of the I/O activity.

14. The method of claim 13, further including performing one of:

pursuant to detecting that the longer EMA of the I/O activity is smaller than the shorter EMA of the I/O activity, adjusting the at least one data migration plan to consider a current up trend change of the I/O activity, and

pursuant to detecting that the longer EMA is larger than the shorter EMA of the I/O activity, adjusting the at least one data migration plan to consider current down trend change of the I/O activity.