Mechanisms for moving data in a hybrid aggregate

• US 8,321,645 B2
• Filed: 04/29/2009
• Issued: 11/27/2012
• Est. Priority Date: 04/29/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Rabii, Faramarz, Strunk, John, Kimmel, Jeffrey S.
• Primary Examiner(s): Hur, J. H.
• Application Number:
  US12/432,649
• Publication Number:
  US 20100281230A1
• Time in Patent Office: 1,308 Days
• Field of Search: 711/103, 711/112, 711/117, 711/165, 711/170, 711/E12.001, 711/E12.002, 711/E12.008
• US Class Current: 711/165
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 12/121 using replacement algorithms
• CPC Class Codes: G06F 2212/214 Solid state disk
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

At least certain embodiments include a method, system and apparatus for relocating data between tiers of storage media in a hybrid storage aggregate encompassing multiple tiers of heterogeneous physical storage media including a file system to automatically relocate the data between tiers. The hybrid storage aggregate includes one or more volumes, each volume including a volume block number space spanning at least a first-tier of storage media and a second tier of storage media of the multiple tiers of heterogeneous physical storage media and the hybrid storage aggregate further includes a control module to cooperatively manage the tiers of the multiple tiers of heterogeneous physical storage media and a file system coupled with the control module, the file system including a policy module configured to make policy decisions based on a set of one or more policies and configured to automatically relocate data between different tiers of the multiple tiers of heterogeneous physical storage media based on the set of policies.

Claims

1. A method comprising:

examining data in a hybrid storage aggregate encompassing multiple tiers of heterogeneous physical storage media using a policy module based on a set of policies that are dynamically adjusted by the policy module to adapt to changes at run-time to improve performance characteristics of the hybrid storage aggregate, the hybrid storage aggregate including a volume having a contiguous volume block number space spanning a first-tier and a second-tier of the multiple tiers of heterogeneous physical storage media;

determining that the data is to be relocated to a different tier of physical storage media based on the set of policies; and

relocating the data to the different tier of physical storage media.

2. The method of claim 1, wherein the first-tier of physical storage media and the second-tier of physical storage media include writable, persistent storage media, and wherein the first-tier of storage media is a lower latency storage media than the second-tier of storage media.

3. The method of claim 2, wherein the first-tier of storage media comprises an electronic storage medium and the second-tier of storage media comprises a magnetic storage medium.

4. The method of claim 3, wherein the electronic storage medium comprises a solid-state device (SSD) and the magnetic storage medium comprises a hard disk device (HDD).

5. The method of claim 1, wherein the examining of the data is performed during read transactions and the changes at run-time include changes in read traffic patterns within the hybrid storage aggregate.

6. The method of claim 5, further comprising determining the data is to be relocated to a lower-latency tier of physical storage media when the data has been one or more of frequently and randomly accessed in read transactions.

7. The method of claim 5, further comprising determining that the data is to be relocated to a higher-latency tier of physical storage media when the data has been either infrequently or sequentially accessed in read transactions.

8. The method of claim 5, wherein the read transactions are write-after-read (WAR) transactions.

9. The method of claim 1, wherein the examining of the data is performed by a segment cleaner module running transparently to a user of the hybrid storage aggregate and the changes at run-time include changes in capacity of storage space among the multiple tiers of heterogeneous physical storage media.

10. The method of claim 9, further comprising:

monitoring the capacity of storage space of each tier of the multiple tiers of heterogeneous physical storage media; and

relocating data from the first-tier of storage media to the second-tier of storage media when the first-tier of storage media approaches full-capacity and the second-tier of storage media has sufficient storage capacity.

11. A method comprising:

dynamically adjusting a set of policies at run-time using a policy module within a storage server to improve performance characteristics of a hybrid storage aggregate, wherein the hybrid storage aggregate includes multiple tiers of heterogeneous storage media, each tier comprising a different storage media technology, the hybrid storage aggregate including a volume having a contiguous volume block number space spanning a first-tier and a second-tier of the multiple tiers of heterogeneous storage media;

examining data in the hybrid storage aggregate using the policy module based on the set of policies;

determining that the data is to be relocated to a different tier of storage media based on the set of policies; and

automatically relocating the data to the different tier of storage media in response to the determining.

12. The method of claim 11, wherein the first-tier of storage media and the second-tier of storage media include writable, persistent storage media, and wherein the first-tier of storage media is a lower latency storage media than the second-tier of storage media.

13. The method of claim 11, wherein examining the data is performed during a read transaction, and wherein the set of policies includes an access-based policy derived from read traffic patterns within the hybrid storage aggregate.

14. The method of claim 13, further comprising:

determining the data is to be relocated to a lower-latency tier of storage media when the data has been one or more of frequently and randomly accessed in the read transactions; and

determining the data is to be relocated to a higher-latency tier of storage media when the data has been either infrequently or sequentially accessed in the read transactions.

15. The method of claim 14, further comprising:

marking a header associated with the block of data in the read transaction as dirty; and

marking the header associated with the block of data as destined to the different tier of storage media.

16. The method of claim 15, further comprising:

marking the header associated with the block of data in the read transaction as fake dirty to update only a physical volume block number associated with the block of data without requiring an update to a virtual volume block number associated with the block of data; and

marking the header associated with the block of data as destined to the different tier of storage media.

17. The method of claim 16, further comprising:

checking the header associated with the block of data to determine if it is marked dirty or fake dirty; and

moving the block of data to the different tier of storage media when the header is marked dirty or fake dirty.

18. The method of claim 11, wherein the examining of the data is performed by a module running transparently to a user of the hybrid storage aggregate and wherein the set of policies includes a capacity-based policy derived from capacity of storage space of the multiple tiers of heterogeneous storage media.

19. The method of claim 18, further comprising:

monitoring the capacity of storage space of the multiple tiers of heterogeneous storage media; and

relocating data from the first-tier of storage media to the second-tier of storage media when the first-tier of storage media approaches full-capacity and the second-tier of storage media has sufficient storage capacity.

20. A storage system comprising:

a processor;

a hybrid storage aggregate encompassing multiple tiers of heterogeneous physical storage media and including a volume having a contiguous volume block number space spanning a first tier and a second tier of the multiple tiers of heterogeneous physical storage media; and

a memory coupled with the processor having a storage manager stored therein, the storage manager operable by the processor to automatically relocate data between tiers of the multiple tiers of heterogeneous physical storage media within the hybrid storage aggregate based on a set of policies that are dynamically adjusted to improve performance characteristics of the hybrid storage aggregate.

21. The storage system of claim 20, wherein the storage manager includes:

a write allocation module configured to perform access-based relocation of data within the hybrid storage aggregate during read operations;

a segment cleaner module running transparently to a user of the hybrid storage aggregate, the segment cleaner module configured to perform capacity-based relocation of data within the hybrid storage aggregate; and

a policy module configured to make policy decisions based on the set of policies stored in the memory and to control operation of the write allocation and segment cleaner modules based on the policy decisions.

22. The storage system of claim 21, wherein the first tier of physical storage media is to store data that is one or more of frequently and randomly accessed and the second-tier of physical storage media is to store data that is either infrequently or sequentially accessed.

23. The storage system of claim 22, wherein the first tier of storage media is a low-latency storage media having performance characteristics superior to the performance characteristics of the second tier of storage media.

24. The storage system of claim 23, wherein the first tier of storage media comprises a writeable, persistent electronic storage media and the second tier of storage media comprises a writeable, persistent magnetic storage media.

25. An apparatus comprising:

a hybrid storage aggregate including;

multiple tiers of heterogeneous storage media, each tier including a different storage media technology, wherein the hybrid storage aggregate includes a volume having a volume block number space spanning a first tier of storage media and a second tier of storage media of the multiple tiers of heterogeneous storage media;

a control module coupled with the hybrid storage aggregate to cooperatively manage the multiple tiers of heterogeneous storage media; and

a policy module coupled with the control module, the policy module configured to make policy decisions based on a set of policies that are dynamically adjusted based on read traffic patterns within the hybrid storage aggregate and to automatically relocate data between different tiers of the multiple tiers of heterogeneous storage media based on the policy decisions.

26. The apparatus of claim 25, wherein the first tier of storage media is a lower-latency storage media than the second tier of storage media.

27. The apparatus of claim 26, wherein the control module is a RAID module.

28. The apparatus of claim 26, further comprising a write allocator, wherein the policy module directs the write allocator to automatically relocate data between tiers of the multiple tiers of heterogeneous storage media during read transactions based on access characteristics of the data.

29. The apparatus of claim 28, wherein the policy module directs the write allocator to automatically relocate data from the second-tier of storage media to the first-tier of storage media when the data is determined to be one or more of frequently and randomly accessed, and to automatically relocate data from the first-tier of storage media to the second-tier of storage media when the data is determined to be either infrequently or sequentially accessed.

30. The apparatus of claim 27, further comprising a segment cleaner, wherein the policy module directs the segment cleaner to automatically relocate data between tiers of the multiple tiers of heterogeneous storage media based on capacity of storage space among the tiers of storage media.

31. The apparatus of claim 30, wherein the policy module directs the segment cleaner to automatically relocate data from the first-tier of storage media to the second-tier of storage media when the first-tier of storage media approaches full-capacity and the second-tier of storage media has sufficient storage capacity.