Method of and system for adaptive selection of a deduplication chunking technique

• US 7,519,635 B1
• Filed: 03/31/2008
• Issued: 04/14/2009
• Est. Priority Date: 03/31/2008
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Haustein, Nils, Troppens, Ulf, Klein, Craig A., Winarski, Daniel J.
• Primary Examiner(s): To; Baoquoc N
• Application Number:
  US12/059,874
• Time in Patent Office: 379 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method of adaptively selecting an optimum data deduplication chunking method receives a request to deduplicate a file, wherein the file has a file type. The method searches a table of file types, wherein the table includes, for each file type, a chunking method, a deduplication ratio, and a depulication ratio threshold. The method selects a chunking method for the file according to the table. The method chunks the file using the selected chunking method. The method deduplicates the chunked file according to prior art deduplication methods. The method calculates a deduplication ratio for the file type and updates the table with the calculated deduplication ratio for the file type. If the calculated deduplication ratio for the file type is less than the deduplication ratio threshold for the file type, the method selects a new chunking method for the file type and updates the table of file types with the new chunking method for the file type.

Claims

1. A method of selecting an optimum deduplication chunking method, which comprises:

receiving a request to deduplicate a file, said file having a file type;

searching a table of file types, said table including for each file type, a chunking method, a deduplication ratio, and a user configurable deduplication ratio threshold;

selecting a chunking method for said file according to said table;

chunking said file using said selected chunking method;

deduplicating said chunked file;

calculating a deduplication ratio for said file type by dividing a sum of individual deduplication ratios achieved for files of said file type by a number of files of said file type deduplicated;

updating said table with said calculated deduplication ratio for said file type;

if said calculated deduplication ratio for said file type is equal to or greater than said deduplication ratio for said file type, maintaining said chunking method for said file type in the said table;

if said calculated deduplication ratio for said file type is less than said deduplication ratio threshold for said file type, selecting a new chunking method for said file type; and

,

updating said table with said new chunking method for said file type.