Preferential selection of candidates for delta compression

• US 9,268,783 B1
• Filed: 03/10/2014
• Issued: 02/23/2016
• Est. Priority Date: 06/13/2012
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Shilane, Philip N., Wallace, Grant R.
• Primary Examiner(s): Choi, Yuk Ting
• Application Number:
  US14/203,359
• Time in Patent Office: 715 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: H03M 7/3091 Data deduplication

Abstract

A computer-implemented method and system for improving efficiency in a delta compression process in a data storage system selects a data chunk to delta compress and selects a set of candidate data chunks using a first selection mechanism. Throughput or resource utilization is monitored. A change is made to a second selection mechanism that increases similarity of the set of candidates with the selected data chunk to improve compression in response to determining high resource availability or high throughput level. A change is made to a third selection mechanism that increases throughput of the delta compression process in response to determining low resources availability or low throughput.

Claims

1. A delta compression system, comprising:

a delta processing module to delta compress a set of data chunks; and

a cache to store a first set of data chunk candidates;

a data storage system to store a second set of data chunk candidates;

a preferential selection module coupled to the cache, data storage system and delta processing module, the preferential selection module configured to;

select, using a selection mechanism, a set of data chunk candidates from at least one of the first and second set of data chunk candidates, wherein the selection mechanism ranks the set of data chunk candidates by similarity and tie-breaks based on location and status information when at least one of resource availability or throughput is higher than a predetermined high threshold value;

monitor delta compression throughput and resource utilization of the delta compression system, and

dynamically adjust the selection mechanism that selects the set of data chunk candidates to be used in delta compression of the set of data chunks to be compressed, in response to detecting changes in the delta compression throughput and resource utilization, wherein the selection mechanism is not adjusted, in response to detecting that the resource availability and throughput are lower than a predetermined high threshold value and in response to detecting that the resource availability and throughput are higher than a predetermined low threshold value.

2. The system of claim 1, wherein the preferential selection module disables delta compression temporarily in response to determining very low throughput or available resources.

3. The system of claim 1, wherein the selection mechanism ranks the set of data chunk candidates by location and status information and tie-breaks based on degree of similarity, in response to detecting that at least one of resource availability or throughput is lower than a predetermined low threshold.

4. The system of claim 3, wherein the status of a data chunk in the set of data chunk candidates comprises whether or not the data chunk is compressed, and an uncompressed data chunk is ranked higher than a compressed data chunk.

5. The system of claim 3, wherein a location comprises the cache or the data storage, and a data chunk in the first set of data chunk candidates that is stored in the cache is ranked higher than a data chunk in the second set of data chunk candidates that is stored in the data storage system.

6. The system of claim 1, wherein the preferential selection module monitors throughput or resource utilization by checking a number of data chunks in a queue to be processed.