Efficient storage tiering

• US 10,042,855 B2
• Filed: 01/26/2016
• Issued: 08/07/2018
• Est. Priority Date: 12/31/2010
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Hsu, Windsor W., Xu, Teng
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): Le, Jessica N
• Application Number:
  US15/007,087
• Publication Number:
  US 20160202911A1
• Time in Patent Office: 924 Days
• Field of Search: 707692
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A method, article of manufacture, and apparatus for transferring data from a source tier to a target tier. In some embodiments, this may include dividing an object stored in the source tier into a plurality of segments, determining if the target tier is missing at least one of the segments, writing the missing segments to the target tier, and deleting the object from the source tier. In some embodiments, the object'"'"'s metadata is updated to indicate that the object is in the target tier.

Claims

1. A method for transferring data from a source tier to a target tier in a data system, comprising:

storing a first object in the source tier wherein storing the first object in the source tier includes dividing the first object into a first plurality of segments using a first deduplication technique and compressing the first plurality of segments using a compression technique of the source tier, and wherein the source tier deduplicates data based at least in part on the first deduplication technique;

determining, using a processor, whether to transfer the first object from the source tier to the target tier, wherein the target tier deduplicates data based at least in part on a second deduplication technique that is different from the first deduplication technique used at the source tier, wherein the determination to transfer the first object is based at least in part on one or more policies for determining whether to transfer an object and a tier including at least one of the source tier and the target tier to which to transfer the object;

decompressing the first plurality of segments using the compression technique of the source tier;

re-segmenting the segmented first object comprising the first plurality of segments into a second plurality of segments based at least in part on the second deduplication technique of the target tier;

compressing a first segment of the second plurality of segments using a compression technique of the target tier, wherein the first segment is not already present at the target tier; and

transferring the compressed first segment to the target tier.

2. The method of claim 1, further comprising updating the first object'"'"'s metadata to indicate that the object is in the target tier.

3. The method of claim 1, further comprising updating the first object'"'"'s metadata to indicate that the target object is in the target tier, wherein updating the first object'"'"'s metadata includes checking if the target object has changed in the source tier since the target object was written to the target tier.

4. The method of claim 1, further comprising verifying that the first object was written to the target tier.

5. The method of claim 1, wherein the data stored in the target tier is accessed more frequently than data stored in the source tier.

6. The method of claim 1, wherein the data stored in the target tier and the data stored in the source tier are accessible to a client over a network.

7. The method of claim 1, further comprising deleting at least one of the first plurality of segments related to the first object from the source tier.

8. A system for transferring data, the system comprising a source tier, a target tier, and a processor configured to:

store a first object in the source tier, wherein storing the first object in the source tier includes dividing the first object into a first plurality of segments using a first deduplication technique and compressing the first plurality of segments using a compression technique of the source tier, and wherein the source tier deduplicates data based at least in part on the first deduplication technique;

determine whether to transfer the first object from the source tier to the target tier, wherein the target tier deduplicates data based at least in part on a second deduplication technique that is different from the first deduplication technique used at the source tier, wherein the determination to transfer the first object is based at least in part on one or more policies for determining whether to transfer an object and a tier including at least one of the source tier and the target tier to which to transfer the object;

decompress the first plurality of segments using the compression technique of the source tier;

re-segmenting the segmented first object comprising the first plurality of segments into a second plurality of segments based at least in part on the second deduplication technique of the target tier;

compress a first segment of the second plurality of segments using a compression technique of the target tier, wherein the first segment is not already present at the target tier; and

transfer the compressed first segment to the target tier.

9. The system of claim 8, the processor is further configured to update the first object'"'"'s metadata to indicate that the object is in the target tier.

10. The system of claim 8, further comprising updating the first object'"'"'s metadata to indicate that the target object is in the target tier, wherein updating the first object'"'"'s metadata includes checking if the target object has changed in the source tier since the target object was written to the target tier.

11. The system of claim 8, the processor is further configured to verify that the first object was written to the target tier.

12. The system of claim 8, wherein the data stored in the target tier is accessed more frequently than data stored in the source tier.

13. The system of claim 8, wherein the data stored in the target tier and the data stored in the source tier are accessible to a client over a network.

14. The system of claim 8, the processor further configured to delete at least one of the first plurality of segments related to the first object from the source tier.

15. A computer program product for transferring data from a source tier to a target tier, comprising a non-transitory computer usable medium having machine readable code embodied therein for:

storing a first object in the source tier, wherein storing the first object in the source tier includes dividing the first object into a first plurality of segments using a first deduplication technique and compressing the first plurality of segments using a compression technique of the source tier, and wherein the source tier deduplicates data based at least in part on a first deduplication technique;

determining whether to transfer the first object from the source tier to the target tier, wherein the target tier deduplicates data based at least in part on a second deduplication technique that is different from the first deduplication technique used at the source tier, wherein the determination to transfer the first object is based at least in part on one or more policies for determining whether to transfer an object and a tier including at least one of the source tier and the target tier to which to transfer the object;

decompressing the first plurality of segments using the compression technique of the source tier;

re-segmenting the segmented first object comprising the first plurality of segments into a second plurality of segments based at least in part on the second deduplication technique of the target tier;

compressing a first segment of the second plurality of segments using a compression technique of the target tier, wherein the first segment is not already present at the target tier; and

transferring the compressed first segment to the target tier.

16. The computer program product of claim 15, further comprising updating the first object'"'"'s metadata to indicate that the first object is in the target tier.

17. The computer program product of claim 15, further comprising verifying that the first object was written to the target tier.

18. The computer program product of claim 15, wherein the data stored in the target tier is accessed more frequently than data stored in the source tier.

19. The computer program product of claim 15, wherein the data stored in the target tier and the data stored in the source tier are accessible to a client over a network.

20. The computer program product of claim 15, further comprising deleting at least one of the first plurality of segments related to the first object from the source tier.