PACKING DEDUPLICATED DATA INTO FINITE-SIZED CONTAINERS

• US 20130339316A1
• Filed: 06/19/2012
• Published: 12/19/2013
• Est. Priority Date: 06/19/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: HIRSCH, Michael, KRAUSE, Thorsten
• Granted Patent: US 9,880,771 B2
• Time in Patent Office: Days
• US Class Current: 707/692
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Deduplicated data is packed into finite-sized containers. A similarity score is calculated between files that are similarly of the deduplicated data. The similarity score is used for grouping the similarly compared files of the deduplicated data into subsets for destaging each of the subsets from a deduplication system to one a finite-sized container.

Claims

1. A method for packing deduplicated data into a plurality of finite-sized containers using a processor device, comprising:

calculating a similarity score between a plurality of similarly compared files of the deduplicated data, and

using the similarity score for grouping the plurality of similarly compared files of the deduplicated data into subsets for destaging each of the subsets from a deduplication system to one of the plurality of finite-sized containers.

2. The method of claim 1, further including comparing previously deduplicated data files in a deduplication system with new data files that are to be deduplicated into the deduplication system at ingestion time for creating the plurality of similarly compared files of the deduplicated data.

3. The method of claim 1, wherein calculating the similarity score further includes, calculating an nth percentage threshold of common data intersections shared between the plurality of similarly compared files of the deduplicated data, wherein a transitive closure between the plurality of similarly compared files of the deduplicated data is determined.

4. The method of claim 3, further including using the transitive closures for assisting with using the similarity score for grouping the plurality of similarly compared files of the deduplicated data into the subsets.

5. The method of claim 4, further including calculating a storage metric value by traversing the each of the subsets for determining a required storage space in one of the plurality of finite-sized containers.

6. The method of claim 3, further including maintaining in a file similarity index an identify of each of the plurality of similarly compared files and the similarity score calculated for each of the plurality of similarly compared files.

7. The method of claim 1, further including receiving an indication by a user which of the plurality of similarly compared files are to be grouped into the subsets for destaging each of the subsets from a deduplication system to one of the plurality of finite-sized containers.

8. A system for packing deduplicated data into a plurality of finite-sized containers in a computing environment, comprising:

a processor device, operable in the computing environment, wherein the at least one processor device is adapted for;

calculating a similarity score between a plurality of similarly compared files of the deduplicated data, and

using the similarity score for grouping the plurality of similarly compared files of the deduplicated data into subsets for destaging each of the subsets from a deduplication system to one of the plurality of finite-sized containers.

9. The system of claim 8, wherein the processor device is further adapted for comparing previously deduplicated data files in a deduplication system with new data files that are to be deduplicated into the deduplication system at ingestion time for creating the plurality of similarly compared files of the deduplicated data.

10. The system of claim 8, wherein the processor device is further adapted for, for calculating the similarity score, calculating an nth percentage threshold of common data intersections shared between the plurality of similarly compared files of the deduplicated data, wherein a transitive closure between the plurality of similarly compared files of the deduplicated data is determined.

11. The system of claim 10, wherein the processor device is further adapted for using the transitive closures for assisting with using the similarity score for grouping the plurality of similarly compared files of the deduplicated data into the subsets.

12. The system of claim 11, wherein the processor device is further adapted for calculating a storage metric value by traversing the each of the subsets for determining a required storage space in one of the plurality of finite-sized containers.

13. The system of claim 10, wherein the processor device is further adapted for maintaining in a file similarity index an identify of each of the plurality of similarly compared files and the similarity score calculated for each of the plurality of similarly compared files.

14. The system of claim 8, wherein the processor device is further adapted for receiving an indication by a user which of the plurality of similarly compared files are to be grouped into the subsets for destaging each of the subsets from a deduplication system to one of the plurality of finite-sized containers.

15. A computer program product for packing deduplicated data into a plurality of finite-sized containers by a processor device, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion for calculating a similarity score between a plurality of similarly compared files of the deduplicated data, and

a second executable portion for using the similarity score for grouping the plurality of similarly compared files of the deduplicated data into subsets for destaging each of the subsets from a deduplication system to one of the plurality of finite-sized containers.

16. The computer program product of claim 15, further including a third executable portion for comparing previously deduplicated data files in a deduplication system with new data files that are to be deduplicated into the deduplication system at ingestion time for creating the plurality of similarly compared files of the deduplicated data.

17. The computer program product of claim 15, wherein calculating the similarity score further includes a third executable portion for calculating an nth percentage threshold of common data intersections shared between the plurality of similarly compared files of the deduplicated data, wherein a transitive closure between the plurality of similarly compared files of the deduplicated data is determined.

18. The computer program product of claim 17, further including a fourth executable portion for using the transitive closures for assisting with using the similarity score for grouping the plurality of similarly compared files of the deduplicated data into the subsets.

19. The computer program product of claim 18, further including a fifth executable portion for calculating a storage metric value by traversing the each of the subsets for determining a required storage space in one of the plurality of finite-sized containers.

20. The computer program product of claim 17, further including a fifth executable portion for maintaining in a file similarity index an identify of each of the plurality of similarly compared files and the similarity score calculated for each of the plurality of similarly compared files.

21. The computer program product of claim 15, further including a fourth executable portion for receiving an indication by a user which of the plurality of similarly compared files are to be grouped into the subsets for destaging each of the subsets from a deduplication system to one of the plurality of finite-sized containers.