EFFICIENT TRANSFER OF DEDUPLICATED DATA

• US 20100036887A1
• Filed: 08/05/2008
• Published: 02/11/2010
• Est. Priority Date: 08/05/2008
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: International Business Machines Corporation
• Inventors: Anglin, Matthew J., Cannon, David M., Martin, Howard N.
• Granted Patent: US 8,788,466 B2
• Time in Patent Office: Days
• US Class Current: N/A
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

One aspect of the present invention includes enabling the efficient transfer of deduplicated data between storage pools in a storage management system without unnecessary reassembly and deduplication of data objects. In one embodiment, the storage management system tracks deduplication information for the data chunks of data objects within an index at the storage management system level, in addition to tracking storage information for each data object within another index at the storage management system level. The data chunk deduplication information is then accessible by any storage pool. Accordingly, transfers of the data objects and data chunks of the data object are easily facilitated, even between non-deduplicating and deduplicating storage pools.

Claims

1. A method in a computer system for performing an efficient transfer of deduplicated data within a storage management system, comprising:

deduplicating a data object within a storage management system into one or more deduplicated data chunks, the storage management system containing a plurality of storage pools;

tracking storage information for the one or more deduplicated data chunks in an index within the storage management system;

tracking deduplication information for the data object in an index within the storage management system;

storing the one or more deduplicated data chunks onto one or more of the plurality of storage pools; and

transferring one or more deduplicated data chunks of the data object from a first storage pool within the plurality of storage pools to a second storage pool within the plurality of storage pools;

wherein each of the indexes used for tracking storage information and tracking deduplication information are accessible by each of the plurality of storage pools.

2. The method as in claim 1, wherein the deduplication information for the data object tracked in an index includes ownership information and a digital signature for each data chunk of the data object.

3. The method as in claim 1, wherein the storage information for the one or more deduplicated data chunks tracked in an index includes references and links to each deduplicated chunk existent within the plurality of storage pools within the storage management system.

4. The method as in claim 1, wherein deduplication of the data object occurs during ingestion of the data object into the storage management system.

5. The method as in claim 1, wherein deduplication of the data object occurs during an operation subsequent to ingestion of the data object into the storage management system.

6. A method in a computer system for performing an efficient transfer of deduplicated data within a storage management system, comprising:

dividing a data object within a storage management system into one or more data chunks;

storing the one or more data chunks onto a plurality of storage pools within the storage management system;

deduplicating the one or more data chunks within one or more of the plurality of storage pools;

tracking deduplication information for each deduplicated data chunk in an index within the storage management system;

tracking storage information for each deduplicated data chunk in an index within the storage management system;

initiating a data transfer of the data object from a source storage pool within the storage management system to a target storage pool within the storage management system;

retrieving deduplication information of the data object from the deduplication information index;

retrieving, if the data object to be transferred from the source storage pool is deduplicated, the storage information for the data object from the storage management index;

transferring, if the target storage pool is a deduplicating pool, necessary data chunks to the target storage pool, including;

adding entries to the storage information index for each of the one or more data chunks; and

transferring each of the one or more data chunks from the source storage pool to the target storage pool that are not existent on the target storage pool; and

transferring, if the target storage pool is a non-deduplicating pool, each of the one or more data chunks from the source storage pool to the target storage pool.

7. A system, comprising:

at least one processor; and

at least one memory which stores instructions operable with the at least one processor for performing an efficient transfer of deduplicated data within a storage management system, the instructions being executed for;

deduplicating a data object within a storage management system into one or more deduplicated data chunks, the storage management system containing a plurality of storage pools;

tracking storage information for the one or more deduplicated data chunks in an index within the storage management system;

tracking deduplication information for the data object in an index within the storage management system;

storing the one or more deduplicated data chunks onto one or more of the plurality of storage pools; and

transferring one or more deduplicated data chunks of the data object from a first storage pool within the plurality of storage pools to a second storage pool within the plurality of storage pools;

wherein each of the indexes used for tracking storage information and tracking deduplication information are accessible by each of the plurality of storage pools.

8. The system of claim 7, wherein the deduplication information for the data object tracked in an index includes ownership information and a digital signature for each data chunk of the data object.

9. The system of claim 7, wherein the storage information for the one or more deduplicated data chunks tracked in an index includes references and links to each deduplicated chunk existent within the plurality of storage pools within the storage management system.

10. The system of claim 7, wherein deduplication of the data object occurs during ingestion of the data object into the storage management system.

11. The system of claim 7, wherein deduplication of the data object occurs during an operation subsequent to ingestion of the data object into the storage management system.

12. A system, comprising:

at least one processor; and

at least one memory which stores instructions operable with the at least one processor for performing an efficient transfer of deduplicated data within a storage management system, the instructions being executed for;

dividing a data object within a storage management system into one or more data chunks;

storing the one or more data chunks onto a plurality of storage pools within the storage management system;

deduplicating the one or more data chunks within one or more of the plurality of storage pools;

tracking deduplication information for each deduplicated data chunk in an index within the storage management system;

tracking storage information for each deduplicated data chunk in an index within the storage management system;

initiating a data transfer of the data object from a source storage pool within the storage management system to a target storage pool within the storage management system;

retrieving deduplication information of the data object from the deduplication information index;

retrieving, if the data object to be transferred from the source storage pool is deduplicated, the storage information for the data object from the storage management index;

transferring, if the target storage pool is a deduplicating pool, necessary data chunks to the target storage pool, including;

adding entries to the storage information index for each of the one or more data chunks; and

transferring each of the one or more data chunks from the source storage pool to the target storage pool that are not existent on the target storage pool; and

transferring, if the target storage pool is a non-deduplicating pool, each of the one or more data chunks from the source storage pool to the target storage pool.

13. A computer program product comprising a computer useable medium having a computer readable program for performing an efficient transfer of deduplicated data within a storage management system, wherein the computer readable program when executed on a computer causes the computer to:

deduplicate a data object within a storage management system into one or more deduplicated data chunks, the storage management system containing a plurality of storage pools;

track storage information for the one or more deduplicated data chunks in an index within the storage management system;

track deduplication information for the data object in an index within the storage management system;

store the one or more deduplicated data chunks onto one or more of the plurality of storage pools; and

transfer the one or more deduplicated data chunks of the data object from a first storage pool within the plurality of storage pools to a second storage pool within the plurality of storage pools;

wherein each of the indexes used for tracking storage information and tracking deduplication information are accessible by each of the plurality of storage pools.

14. The computer program product of claim 13, wherein the deduplication information for the data object tracked in an index includes ownership information and a digital signature for each data chunk of the data object.

15. The computer program product of claim 13, wherein the storage information for the one or more deduplicated data chunks tracked in an index includes references and links to each deduplicated chunk existent within the plurality of storage pools within the storage management system.

16. The computer program product of claim 13, wherein deduplication of the data object occurs during ingestion of the data object into the storage management system.

17. The computer program product of claim 13, wherein deduplication of the data object occurs during an operation subsequent to ingestion of the data object into the storage management system.

18. A computer program product comprising a computer useable medium having a computer readable program for performing an efficient transfer of deduplicated data within a storage management system, wherein the computer readable program when executed on a computer causes the computer to:

divide a data object within a storage management system into one or more data chunks;

store the one or more data chunks onto a plurality of storage pools within the storage management system;

deduplicate the one or more data chunks within one or more of the plurality of storage pools;

track deduplication information for each deduplicated data chunk in an index within the storage management system;

track storage information for each deduplicated data chunk in an index within the storage management system;

initiate a data transfer of the data object from a source storage pool within the storage management system to a target storage pool within the storage management system;

retrieve deduplication information of the data object from the deduplication information index;

retrieve, if the data object to be transferred from the source storage pool is deduplicated, the storage information for the data object from the storage management index;

transfer, if the target storage pool is a deduplicating pool, necessary data chunks to the target storage pool, including;

adding entries to the storage information index for each of the one or more data chunks; and

transferring each of the one or more data chunks from the source storage pool to the target storage pool that are not existent on the target storage pool; and

transfer, if the target storage pool is a non-deduplicating pool, each of the one or more data chunks from the source storage pool to the target storage pool.