POLICY-BASED SHARING OF REDUNDANT DATA ACROSS STORAGE POOLS IN A DEDUPLICATING SYSTEM

• US 20100082558A1
• Filed: 10/01/2008
• Published: 04/01/2010
• Est. Priority Date: 10/01/2008
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Anglin, Matthew J., Cannon, David M.
• Granted Patent: US 8,495,032 B2
• Time in Patent Office: Days
• US Class Current: 707/694
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

One aspect of the present invention includes enabling data chunks to be shared among different storage pools within a storage management system, according the use of deduplication and storage information kept at the system level, and applied with policy-based rules that define the scope of deduplication. In one embodiment, the parameters of performing deduplication are defined within the policy, particularly which of the plurality of storage pools allow deduplication to which other pools. Accordingly, a data object may be linked to deduplicated data chunks existent within other storage pools, and the transfer of a data object may occur by simply creating references to existing data chunks in other pools provided the policy allows the pool to reference chunks in these other pools. Additionally, a group of storage pools may be defined within the policy to perform a common set of deduplication activities across all pools within the group.

Claims

1. A method in a computer system for implementing policy-based sharing of redundant data across deduplicating storage pools within a storage management system, comprising:

dividing a data object within a storage management system into one or more deduplicated data chunks, the storage management system containing a plurality of storage pools;

tracking storage information for the one or more deduplicated data chunks in an index within the storage management system;

tracking deduplication information for the data object in an index within the storage management system;

enabling deduplication of the data object across more than one of the plurality of storage pools based on a defined policy; and

deduplicating the data object across more than one of the plurality of storage pools through use of the storage information and deduplication information according to the defined policy, including creating references to one or more of the deduplicated data chunks existent on other storage pools within the plurality of storage pools;

wherein each of the indexes used for tracking storage information and tracking deduplication information are accessible by each of the plurality of storage pools.

2. The method as in claim 1, wherein the policy controls which storage pools within the plurality of storage pools may create references to data chunks stored in the other storage pools within the plurality of storage pools, the other storage pools specified by the policy.

3. The method as in claim 1, wherein the policy defines rules for a pool group comprising one or more storage pools within the plurality of storage pools.

4. The method as in claim 3, wherein the policy is configured to enable references from a defined storage pool to data chunks stored within pools which share group membership with the defined storage pool, and to disable references from the defined storage pool to data chunks stored within pools which do not share group membership with the defined storage pool.

5. The method as in claim 1, further comprising transferring the data object from a source storage pool to a target storage pool within the plurality of storage pools wherein, for each data chunk of the data object, the data chunk is transferred only if the data chunk is not stored within either of the target storage pool or another storage pool within the plurality of storage pools which the target storage pool can reference according to the policy.

6. The method as in claim 1, wherein the policy defines rules for client node groups, wherein client nodes within a defined node group can share data chunks regardless of storage location of the data chunks within the plurality of storage pools, and wherein client nodes not within the defined node group cannot share data chunks.

7. The method as in claim 1, wherein the policy is derived automatically by the storage management system, and wherein the policy may be explicitly overridden or complemented by defined policy settings.

8. A system, comprising:

at least one processor; and

at least one memory which stores instructions operable with the at least one processor for implementing policy-based sharing of redundant data across deduplicating storage pools within a storage management system, the instructions being executed for;

dividing a data object within a storage management system into one or more deduplicated data chunks, the storage management system containing a plurality of storage pools;

tracking storage information for the one or more deduplicated data chunks in an index within the storage management system;

tracking deduplication information for the data object in an index within the storage management system;

enabling deduplication of the data object across more than one of the plurality of storage pools based on a defined policy; and

deduplicating the data object across more than one of the plurality of storage pools through use of the storage information and deduplication information according to the defined policy, including creating references to one or more of the deduplicated data chunks existent on other storage pools within the plurality of storage pools;

wherein each of the indexes used for tracking storage information and tracking deduplication information are accessible by each of the plurality of storage pools.

9. The system of claim 8, wherein the policy controls which storage pools within the plurality of storage pools may create references to data chunks stored in the other storage pools within the plurality of storage pools, the other storage pools specified by the policy.

10. The system of claim 8, wherein the policy defines rules for a pool group comprising one or more storage pools within the plurality of storage pools.

11. The system of claim 10, wherein the policy is configured to enable references from a defined storage pool to data chunks stored within pools which share group membership with the defined storage pool, and to disable references from the defined storage pool to data chunks stored within pools which do not share group membership with the defined storage pool.

12. The system of claim 8, further comprising transferring the data object from a source storage pool to a target storage pool within the plurality of storage pools wherein, for each data chunk of the data object, the data chunk is transferred only if the data chunk is not stored within either of the target storage pool or another storage pool within the plurality of storage pools which the target storage pool can reference according to the policy.

13. The system of claim 8, wherein the policy defines rules for client node groups, wherein client nodes within a defined node group can share data chunks regardless of storage location of the data chunks within the plurality of storage pools, and wherein client nodes not within the defined node group cannot share data chunks.

14. The system of claim 8, wherein the policy is derived automatically by the storage management system, and wherein the policy may be explicitly overridden or complemented by defined policy settings.

15. A computer program product comprising a computer useable medium having a computer readable program for implementing policy-based sharing of redundant data across deduplicating storage pools within a storage management system, wherein the computer readable program when executed on a computer causes the computer to:

divide a data object within a storage management system into one or more deduplicated data chunks, the storage management system containing a plurality of storage pools;

track storage information for the one or more deduplicated data chunks in an index within the storage management system;

track deduplication information for the data object in an index within the storage management system;

enable deduplication of the data object across more than one of the plurality of storage pools based on a defined policy; and

deduplicate the data object across more than one of the plurality of storage pools through use of the storage information and deduplication information according to the defined policy, including creating references to one or more of the deduplicated data chunks existent on other storage pools within the plurality of storage pools;

wherein each of the indexes used for tracking storage information and tracking deduplication information are accessible by each of the plurality of storage pools.

16. The computer program product of claim 15, wherein the policy controls which storage pools within the plurality of storage pools may create references to data chunks stored in the other storage pools within the plurality of storage pools, the other storage pools specified by the policy.

17. The computer program product of claim 15, wherein the policy defines rules for a pool group comprising one or more storage pools within the plurality of storage pools.

18. The computer program product of claim 17, wherein the policy is configured to enable references from a defined storage pool to data chunks stored within pools which share group membership with the defined storage pool, and to disable references from the defined storage pool to data chunks stored within pools which do not share group membership with the defined storage pool.

19. The computer program product of claim 15, further comprising transferring the data object from a source storage pool to a target storage pool within the plurality of storage pools wherein, for each data chunk of the data object, the data chunk is transferred only if the data chunk is not stored within either of the target storage pool or another storage pool within the plurality of storage pools which the target storage pool can reference according to the policy.

20. The computer program product of claim 15, wherein the policy defines rules for client node groups, wherein client nodes within a defined node group can share data chunks regardless of storage location of the data chunks within the plurality of storage pools, and wherein client nodes not within the defined node group cannot share data chunks.

21. The computer program product of claim 15, wherein the policy is derived automatically by the storage management system, and wherein the policy may be explicitly overridden or complemented by defined policy settings.