Performing data storage optimizations across multiple data storage systems

• US 8,935,493 B1
• Filed: 05/08/2012
• Issued: 01/13/2015
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Dolan, Sean C., Naamad, Amnon, Marshak, Marik, Veprinsky, Alexandr, Sahin, Adnan, Wang, Hui, Liu, Xiaomei, Aharoni, Dan, Kirac, Ahmet, Martin, Owen
• Primary Examiner(s): RUTZ, JARED IAN
• Application Number:
  US13/466,775
• Time in Patent Office: 980 Days
• Field of Search: 711/117, 711/162
• US Class Current: 711/161
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/068 Hybrid storage device

Abstract

Described are techniques for performing data storage optimizations. A data storage optimizer on a first data storage system connected to a set of other data storage systems receives performance classification information. The first data storage system includes a first set of storage devices of a first performance classification. The set of other data storage systems includes a second set of storage devices of a second performance classification different from the first performance classification. A write operation is received from the host to write first data to a target address of a virtually provisioned device. It is determined that physical storage is not currently allocated for the target address of the virtually provisioned device. A first portion of physical storage is allocated for storing the first data. The first portion of physical storage is included in a storage device of the second set having the second performance classification.

Claims

1. A method for performing data storage optimizations comprising:

receiving, by a data storage optimizer on a first data storage system connected to a set of one or more other data storage systems, performance classification information, said first data storage system including a first set of one or more storage devices of a first performance classification and said set of one or more other data storage systems including a second set of one or more storage devices of a second performance classification different from the first performance classification, wherein said performance classification information indicates that storage devices of the second set are of the second performance classification;

receiving, by the first data storage system from a host, a write operation to write first data to a target address of a virtually provisioned device;

determining that physical storage is not currently allocated for the target address of the virtually provisioned device; and

allocating a first portion of physical storage for storing the first data, the first portion of physical storage being included in a storage device of the second set having the second performance classification, wherein said target address is included in a first extent of the virtually provisioned device and said second performance classification is selected from a plurality of performance classifications in accordance with selection criteria, said selection criteria including one or more extent-level metrics characterizing workload of data operations directed to the first extent.

2. The method of claim 1, wherein a subrange of a logical address range for the virtually provisioned device includes the target address and is mapped to the first portion of physical storage, and wherein said allocating includes selecting said second performance classification from the plurality of performance classifications in accordance with said selection criteria, said selection criteria including a first set of one or more metrics used to characterize an expected workload or level of activity in connection with data operations directed to said subrange.

3. The method of claim 2, wherein the first set of one or more metrics includes at least one metric based on any of long term and short term statistical information collected for a second subrange of the logical address range of the virtually provisioned device, and wherein said second subrange includes said subrange.

4. The method of claim 3, wherein the selection criteria includes a score calculated using said first set of one or more metrics and wherein the first set includes a metric weighted based on an expected I/O size for data operations directed to the second subrange.

5. The method of claim 2, further comprising:

performing processing to collect one or more metrics for each of a plurality of data portions of said virtually provisioned device; and

performing processing for evaluating whether to perform a data movement optimization to move the first data currently included in one of the plurality of data portions of the virtually provisioned device and currently stored on the first portion of physical storage of the storage device having the second performance classification to a second portion of physical storage included on another device of the first set having the first performance classification, wherein said processing for evaluating uses information including at least one said one or more metrics collected for each data portion of the virtually provisioned device, wherein said data movement optimization is a promotion if the first performance classification is associated with a higher level of performance than the second performance classification, and said data movement optimization is otherwise a demotion if the first performance classification is associated with a lower level of performance than said second performance classification.

6. The method of claim 5, wherein said plurality of data portions are extents, the first extent being included in said extents, each of said extents being partitioned into a plurality of sub extents, said target address of the virtually provisioned device being included in a first sub extent of the first extent.

7. The method of claim 2, wherein each of said plurality of performance classifications corresponds to a different storage tier of physical drives.

8. The method of claim 7, wherein said first performance classification corresponds to a first storage tier of physical storage drives having a first set of performance characteristics and being of a first drive technology and wherein said second performance classification corresponds to a second storage tier of physical drives having a second set of performance characteristics and being of a second drive technology different from the first drive technology.

9. The method of claim 1, further comprising:

performing, by the first data storage system, discovery processing to obtain said performance classification information about said second set of devices of the set of one or more other data storage systems.

10. The method of claim 9, wherein devices in the second set are logical devices and said discovery processing includes:

issuing a plurality of I/O operations to the logical devices;

determining response times of the logical devices in response to issuing the plurality of I/O operations; and

determining, in accordance with said response times, one of a plurality of performance classifications for each of the logical devices having storage provisioned from one or more physical devices of the one or more other data storage systems, said performance classification for said each logical device being a performance classification of the one or more physical devices upon which data of said each logical device is stored.

11. The method of claim 9, wherein said second set of one or more devices includes a plurality of logical devices and said discovery processing includes:

receiving, for the plurality of logical devices, a set of baseline response times obtained when issuing data operations to the plurality of logical devices, each of said plurality of logical devices having storage provisioned on one or more physical devices;

selecting a first of the plurality of logical devices, wherein a remaining set of logical devices includes each of the plurality of logical devices other than the first logical device;

issuing data operations to the first logical device at a first rate while also issuing data operations to each logical device of the remaining set, wherein said first rate is sufficient to result in a first observed response time for the first logical device that is significantly more than a corresponding one of the baseline response times for the first logical device, and wherein said first rate is sufficient to result in an observed response time for each logical device of the remaining set having a portion of data stored on a same physical device as a portion of data of the first logical device whereby the observed response time for said each logical device of the remaining set is significantly more than a corresponding one of the baseline response times for said each logical device;

receiving a set of observed response times for the remaining set of logical devices;

determining, for each logical device of the remaining set of logical devices, whether said each logical device has a corresponding one of the observed response times of the set that is significantly more than a corresponding one of the baseline response times for said each logical device; and

determining a first logical device set of one or more logical devices having physical device dependencies, said first logical device set including the first logical device and each logical device of the remaining set determined to have a corresponding observed response time of the set of observed response times that is significantly greater than a corresponding one of the baseline response times for said each logical device.

12. The method of claim 1, wherein the storage device of the second set is included in a second storage system of the set of one or more other data storage systems, and wherein the storage device of the second data storage system is accessible to the first data storage system through a path from a first port of a backend disk adapter of the first data storage system as an initiator to a second port of a front end adapter of the second data storage system.

13. The method of claim 1, wherein said data storage optimizer performs processing to determine whether to automatically perform compression of one or more data portions of the second set of storage devices included in the one or more other data storage systems.

14. The method of claim 13, wherein a first data portion of the second set of storage devices is compressed if it is determined that the first data portion is a candidate for compression in accordance with compression criteria, said compression criteria including a time compression threshold indicating a minimum amount of time that a data portion has been deemed idle whereby the first data portion is a candidate for compression if the first data portion has a level of inactivity greater than the time compression threshold.

15. The method of claim 1, wherein said data storage optimizer performs processing to determine whether to automatically decompress one or more data portions of the second set of storage devices included in the one or more other data storage systems currently stored in a compressed form, whereby a data portion is decompressed if the data portion has a sufficient level of activity.

16. A method for performing data storage optimizations comprising:

receiving, by a data storage optimizer on a first data storage system connected to a set of one or more other data storage systems, performance classification information, said first data storage system including a first set of one or more storage devices of a first performance classification and said set of one or more other data storage systems including a second set of one or more storage devices of a second performance classification different from the first performance classification, wherein said performance classification information indicates that storage devices of the second set are of the second performance classification;

receiving, by the first data storage system from a host, a write operation to write first data to a target address of a virtually provisioned device;

determining that physical storage is not currently allocated for the target address of the virtually provisioned device; and

allocating a first portion of physical storage for storing the first data, the first portion of physical storage being included in a storage device of the second set having the second performance classification, wherein a subrange of a logical address range for the virtually provisioned device includes the target address and is mapped to the first portion of physical storage, and wherein said allocating includes selecting said second performance classification from a plurality of performance classifications in accordance with selection criteria, said selection criteria including a first set of one or more metrics used to characterize an expected workload or level of activity in connection with data operations directed to said subrange, and wherein the method further includes;

performing processing to collect one or more metrics for each of a plurality of data portions of said virtually provisioned device; and

performing processing for evaluating whether to perform a data movement optimization to move the first data currently included in one of the plurality of data portions of the virtually provisioned device and currently stored on the first portion of physical storage of the storage device having the second performance classification to a second portion of physical storage included on another device of the first set having the first performance classification, wherein said processing for evaluating uses information including at least one said one or more metrics collected for each data portion of the virtually provisioned device, wherein said plurality of data portions are extents, each of said extents being partitioned into a plurality of sub extents, said target address of the virtually provisioned device being included in a first sub extent of a first of the extents, wherein said one or more metrics for each of said extents includes one or more extent-level metrics, wherein one or more thresholds identify performance score thresholds and are dynamically determined using any of a capacity limit and a performance limit, and wherein said selecting said second performance classification in accordance with selection criteria includes;

receiving first extent-level metrics for the first extent;

determining a first score using said first extent-level metrics for the first extent; and

selecting, using information including said first score and said one or more thresholds, said second performance classification from said plurality of performance classifications.

17. The method of claim 16, wherein said first set of one or more metrics includes an activity map comprising a value for each sub extent in each of the plurality of extents, a first activity map identifying a level of activity for said first sub extent including the target address of the virtually provisioned device.

18. A system comprising:

a plurality of data storage systems including a first data storage system and one or more other data storage systems, said first data storage system including a first set of one or more storage devices of a first performance classification and said one or more other data storage systems including a second set of one or more storage devices of a second performance classification different from the first performance classification; and

a computer readable medium comprising code stored thereon for performing data storage optimizations, the computer readable medium comprising code stored thereon that;

receives performance classification information by a data storage optimizer on the first data storage system connected to the one or more other data storage systems, wherein said performance classification information indicates that storage devices of the second set are of the second performance classification;

receives, by the first data storage system from a host, a write operation to write first data to a target address of a virtually provisioned device;

determines that physical storage is not currently allocated for the target address of the virtually provisioned device; and

allocates a first portion of physical storage for storing the first data, the first portion of physical storage being included in a storage device of the second set having the second performance classification, wherein said target address is included in a first extent of the virtually provisioned device and said second performance classification is selected from a plurality of performance classifications in accordance with selection criteria, said selection criteria including one or more extent-level metrics characterizing workload of data operations directed to the first extent.

19. A system comprising:

a plurality of data storage systems including a first data storage system and one or more other data storage systems, said first data storage system including a first set of one or more storage devices of a first performance classification and said one or more other data storage systems including a second set of one or more storage devices of a second performance classification different from the first performance classification; and

a computer readable medium comprising code stored thereon for performing data storage optimizations, the computer readable medium comprising code stored thereon that;

receives performance classification information by a data storage optimizer on the first data storage system connected to the one or more other data storage systems, wherein said performance classification information indicates that storage devices of the second set are of the second performance classification;

receives, by the first data storage system from a host, a write operation to write first data to a target address of a virtually provisioned device;

determines that physical storage is not currently allocated for the target address of the virtually provisioned device; and

allocates a first portion of physical storage for storing the first data, the first portion of physical storage being included in a storage device of the second set having the second performance classification, wherein said second set of devices is a set of logical devices and the computer readable medium further comprises;

code that performs discovery processing to determine said performance classification information, where said discovery processing includes issuing I/Os to said set of logical devices and determining performance classifications based on observed response times; and

code that evaluates data movement alternatives, wherein said code that evaluates data movement alternatives further includes code that;

receives a set of criteria including capacity limits and performance limits, said performance limits including a plurality of sets of performance limits, each of said plurality of sets specifying a plurality of performance limits for a plurality of storage tiers, a first of said plurality of storage tiers corresponding to said first performance classification and a second of said plurality of storage tiers corresponding to said second performance classification;

performs first processing to evaluate a plurality of alternatives for use in data movement with respect to said set of logical devices having data stored on a set of physical storage devices, each of said plurality of alternatives including a different set of data movement criteria comprising said capacity limits and a different one of said plurality of sets of performance limits, said set of physical storage devices comprising at least a first physical device of one of said plurality of storage tiers and a second physical device of another one of said plurality of storage tiers; and

selects one of said plurality of sets of performance limits in accordance with said first processing.