DEDUPLICATING HYBRID STORAGE AGGREGATE

• US 20130238832A1
• Filed: 03/07/2012
• Published: 09/12/2013
• Est. Priority Date: 03/07/2012
• Status: Abandoned Application

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Dronamraju, Ravikanth, Sundaram, Rajesh, Doucette, Douglas P.
• Application Number:
  US13/413,898
• Publication Number:
  US 20130238832A1
• Time in Patent Office: Days
• US Class Current: 711/103
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/217 Hybrid disk, e.g. using both magnetic and solid state storage devices
• CPC Class Codes: G06F 2212/461 Sector or disk block
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

Methods and apparatuses for performing deduplication in a hybrid storage aggregate are provided. In one example, a method includes operating a hybrid storage aggregate that includes a plurality of tiers of different types of physical storage media. The method includes identifying a first storage block and a second storage block of the hybrid storage aggregate that contain identical data and identifying caching statuses of the first storage block and the second storage block. The method also includes deduplicating the first storage block and the second storage block based on the caching statuses of the first storage block and the second storage block.

Claims

1. A method comprising:

operating a hybrid storage aggregate that includes a plurality of tiers of different types of physical storage media;

identifying a first storage block and a second storage block of the hybrid storage aggregate that contain identical data;

identifying caching statuses of the first storage block and the second storage block; and

deduplicating the first storage block and the second storage block based on the caching statuses of the first storage block and the second storage block.

2. The method of claim 1 wherein a first tier of storage of the plurality of tiers includes persistent storage media having a lower latency than persistent storage media of a second tier of storage of the plurality of tiers.

3. The method of claim 2 wherein the persistent storage media of the first tier of storage includes a solid state storage device and the persistent storage media of the second tier of storage includes a disk based storage device.

4. The method of claim 2 further comprising operating the first tier of storage as a cache for the second tier of storage.

5. The method of claim 2 wherein a third tier of storage of the plurality of tiers includes storage media having a lower latency than the persistent storage media of the first tier of storage and further comprising operating the third tier of storage as a cache for one or more of the first and the second tiers of storage.

6. The method of claim 2 wherein:

the first and the second storage blocks are located in the second tier of storage;

a third storage block located in the first tier of storage contains data identical to the data of the first storage block and metadata associated with the first storage block points to the third storage block; and

deduplicating includes changing metadata associated with the second storage block to point to the first storage block.

7. The method of claim 6 further comprising:

receiving a request to read the second storage block; and

transmitting the data of the third storage block in response to the request.

8. The method of claim 2 wherein:

the first tier of storage is operated as a cache for the second tier of storage;

the first and the second storage blocks are located in the second tier of storage;

a third storage block located in the first tier of storage contains data identical to the data of the first storage block and metadata associated with the first storage block points to the third storage block; and

deduplicating includes changing metadata associated with the third storage block to point to the second storage block and changing metadata associated with the first storage block to point to the second storage block.

9. The method of claim 8 further comprising:

receiving a request to read the first storage block; and

transmitting the data of the third storage block in response to the request.

10. The method of claim 2 wherein:

the first tier of storage is operated as a cache for the second tier of storage;

the first storage block is located in the first tier of storage and has an access frequency below a threshold;

the second storage block is located in the second tier of storage; and

deduplicating includes changing metadata of the second storage block to point to the first storage block to make the first storage block a read cache for the second storage block.

11. The method of claim 2 wherein:

the first tier of storage is operated as a cache for the second tier of storage;

the first storage block and the second storage block are located in the first tier of storage;

a first reference count indicates a number of files which use the first storage block and a second reference count indicates a number of files which use the second storage block, wherein the first reference count is greater than the second reference count; and

deduplicating includes;

changing metadata of the second storage block to point to the first storage block; and

adding an access frequency of the second storage block to an access frequency of the first storage block.

12. A storage server system comprising:

a processor; and

a memory coupled with the processor and including a storage manager that directs the processor to;

operate a hybrid storage aggregate including a first tier of storage and a second tier of storage, wherein the first tier or storage has a lower latency than the second tier of storage;

identify a first storage block and a second storage block in the hybrid storage aggregate that contain duplicate data;

identify caching relationships associated with the first storage block and the second storage block; and

deduplicate the first and the second storage blocks based on the caching relationships.

13. The storage server system of claim 12 wherein persistent storage media of the first tier of storage includes a solid state device and persistent storage media of the second tier of storage includes a hard disk device.

14. The storage server system of claim 12 wherein the storage manager further directs the processor to operate the first tier of storage as a cache for the second tier of storage.

15. The storage server system of claim 12 wherein the hybrid storage aggregate includes a third tier of storage having a lower latency than the first tier of storage and the storage manager further directs the processor to operate the third tier of storage as a cache for one or more of the first and the second tiers of storage.

16. The storage server system of claim 12 wherein:

the storage manager further directs the processor to operate the first tier of storage as a cache for the second tier of storage;

the first and the second storage blocks are located in the second tier of storage;

the first storage block is read cached by a third storage block located in the first tier of storage that contains data identical to the data of the first storage block and metadata associated with the first storage block points to the third storage block; and

deduplicating includes changing metadata associated with the second storage block to point to the first storage block.

17. The storage server system of claim 16 wherein the storage manager further directs the processor to:

receive a request to read the second storage block; and

transmit the data of the third storage block in response to the request.

18. The storage server system of claim 12 wherein:

the storage manager further directs the processor to operate the first tier of storage as a cache for the second tier of storage;

the first and the second storage blocks are located in the second tier of storage;

the first and the second storage blocks are located in the second tier of storage;

the first storage block is read cached by a third storage block located in the first tier of storage that contains data identical to the data of the first storage block and metadata associated with the first storage block points to the third storage block; and

deduplicating includes changing metadata associated with the third storage block to point to the second storage block and changing metadata associated with the first storage block to point to the second storage block.

19. The storage server system of claim 18 wherein the storage manager further directs the processor to:

receive a request to read the first storage block; and

transmit the data of the third storage block in response to the request.

20. The storage server system of claim 12 wherein:

the storage manager further directs the processor to operate the first tier of storage as a cache for the second tier of storage;

the first storage block is located in the first tier of storage and has an access frequency below a threshold;

the second storage block is located in the second tier of storage; and

deduplicating includes changing metadata of the second storage block to point to the first storage block to make the first storage block a read cache for the second storage block.

21. The storage server system of claim 12 wherein:

the storage manager further directs the processor to operate the first tier of storage as a cache for the second tier of storage;

the first storage block and the second storage block are located in the first tier of storage;

a first reference count indicates a number of files which use the first storage block and a second reference count indicates a number of files which use the second storage block, wherein the first reference count is greater than the second reference count; and

deduplicating includes;

changing metadata of the second storage block to point to the first storage block; and

adding an access frequency of the second storage block to an access frequency of the first storage block.

22. A non-transitory machine-readable medium comprising non-transitory instructions that, when executed by one or more processors, direct the one or more processors to:

identify a first storage block and a second storage block that contain identical data, the first storage block and the second storage block both located in a hybrid storage aggregate that includes a first tier of storage and a second tier of storage wherein the first tier or storage has a lower latency than the second tier of storage and the first tier of storage is operated as a cache for the second tier of storage;

identify caching statuses associated with the first storage block and the second storage block; and

deduplicate the first and the second storage blocks based on the caching statuses.

23. The non-transitory machine-readable medium of claim 22 wherein persistent storage media of the first tier of storage includes a solid state device and persistent storage media of the second tier of storage includes a hard disk device.

24. The non-transitory machine-readable medium of claim 22 wherein the hybrid storage aggregate includes a third tier of storage having a lower latency than the first tier of storage and the instructions further direct the one or more processors to operate the third tier of storage as a cache for one or more of the first and the second tiers of storage.

25. The non-transitory machine-readable medium of claim 22 wherein:

the first and the second storage blocks are located in the second tier of storage;

a third storage block located in the first tier of storage contains data identical to the data of the first storage block and metadata associated with the first storage block points to the third storage block; and

deduplicating includes changing metadata associated with the second storage block to point to the first storage block.

26. The non-transitory machine-readable medium of claim 25 wherein the storage manager further directs the processor to:

receive a request to read the second storage block; and

transmit the data of the third storage block in response to the request.

27. The non-transitory machine-readable medium of claim 22 wherein:

the first and the second storage blocks are located in the second tier of storage;

a third storage block located in the first tier of storage contains data identical to the data of the first storage block and metadata associated with the first storage block points to the third storage block; and

deduplicating includes changing metadata associated with the third storage block to point to the second storage block and changing metadata associated with the first storage block to point to the second storage block.

28. The non-transitory machine-readable medium of claim 27 wherein the instructions further direct the one or more processors to:

receive a request to read the first storage block; and

transmit the data of the third storage block in response to the request.

29. The non-transitory machine-readable medium of claim 22 wherein:

the first storage block is located in the first tier of storage and has an access frequency below a threshold;

the second storage block is located in the second tier of storage; and

deduplicating includes changing metadata of the second storage block to point to the first storage block to make the first storage block a read cache for the second storage block.

30. The non-transitory machine-readable medium storage of claim 22 wherein:

the first storage block and the second storage block are located in the first tier of storage;

a first reference count indicates a number of files which use the first storage block and a second reference count indicates a number of files which use the second storage block, wherein the first reference count is greater than the second reference count; and

deduplicating includes;

changing metadata of the second storage block to point to the first storage block; and

adding an access frequency of the second storage block to an access frequency of the first storage block.