Overwriting part of compressed data without decompressing on-disk compressed data

• US 9,477,420 B2
• Filed: 05/23/2014
• Issued: 10/25/2016
• Est. Priority Date: 05/02/2011
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Yadav, Sandeep, Faith, Rickard E., Periyagaram, Subramaniam V., Lewis, Blake H., Prakash, Ashish
• Primary Examiner(s): Yi, David X
• Assistant Examiner(s): RANKIN, CANDICE A
• Application Number:
  US14/286,900
• Publication Number:
  US 20140258648A1
• Time in Patent Office: 886 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 2211/1014 Compression, i.e. RAID systems with parity using compression techniques
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Partially overwriting a compression group without decompressing compressed data can consumption of resources for the decompression. A storage server partially overwrites the compression group when a file block identifier of a client'"'"'s write request resolves to the compression group. The compression group remains compressed while the partial overwriting is performed.

Claims

1. A method comprising:

in response to receiving a write request that indicates a first file block identifier and indicates data to be written, determining that the first file block identifier resolves to a data block in a compression group, the compression group comprising a group of data blocks that is compressed; and

partially overwriting the compression group while the compression group remains compressed, wherein the partial overwriting comprises,

storing the data of the write request as uncompressed data outside the compression group, and

updating references used for resolving file block identifiers of the file to data blocks of the file, wherein the updating comprises updating the references to map the first file block identifier to the compression group and to the uncompressed data;

determining whether a predetermined criteria is violated by the partial overwriting of the compression group, wherein the predetermined criteria is selected from a group consisting of a system usage guarantee, a space usage guarantee, and a user-defined criteria; and

in response to determining that the predetermined criteria is violated, compressing the partially overwritten compression group.

2. The method of claim 1, wherein updating the references comprises adding an indirection reference to an indirection block of the file, wherein the indirection block includes a first reference to the data blocks of the compression group and the added reference references the uncompressed data.

3. The method of claim 1, wherein partially overwriting comprises:

allocating an extent identifier;

determining an offset into an existing extent associated with the compression group, the offset comprising a difference between a first block number provided in the write request and a second block number for a start of the existing extent, wherein the first file block identifier is the first block number; and

in response to determining that the offset is non-zero,

generating a first extent entry having the allocated extent identifier and a reference to a start of the compression group,

generating a second extent entry having a reference to the uncompressed data of the write request,

generating a third extent entry having a reference to a remainder portion of the compression group that is not overwritten by the uncompressed data, and

associating the first extent entry, the second extent entry, and the third extent entry with the first block number.

4. The method of claim 1, further comprising:

determining whether the partially overwritten compression group including the uncompressed data should be compressed;

deferring compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed; and

compressing the partially overwritten compression group if the partially overwritten compression group should be compressed.

5. The method of claim 4, wherein the determining whether the partially overwritten compression group including the uncompressed data should be compressed comprises:

determining whether a snapshot of data is to occur within a predetermined amount of time;

determining that the partially overwritten compression group should be compressed if the snapshot of data is to occur within a predetermined amount of time; and

determining that the partially overwritten compression group should not be compressed if the snapshot of data is not to occur within a predetermined amount of time.

6. The method of claim 4, wherein the determining whether the partially overwritten compression group should be compressed comprises:

determining whether the data in the compression group is cold data;

determining that the partially overwritten compression group should be compressed if the data in the compression group is cold data; and

determining that the partially overwritten compression group should not be compressed if the data in the compression group is not cold data.

7. A system comprising:

one or more processors; and

a non-transitory computer-readable storage medium communicably coupled to the one or more processors, the non-transitory computer-readable storage medium having stored thereon instructions executable by the one or more processors to cause the system to;

in response to receipt of a write request that indicates a file block identifier and that indicates data to be written, determine whether the file block identifier resolves to a compression group, the compression group comprising a group of data blocks that is compressed; and

partially overwrite the compression group while the compression group remains compressed, wherein the instructions to partially overwrite comprise instructions executable by the processor to cause the system to,

store the data of the write request as uncompressed data outside the compression group, and

update references used for resolving file block identifiers of the file to data blocks of the file, wherein the instructions to update comprise instructions executable by the processor to cause the system to update the references to map the file block identifier to the compression group and to the uncompressed data;

determine whether a predetermined criteria is violated by the partial overwriting of the compression group, wherein the predetermined criteria is selected from a group consisting of a system usage guarantee, a space usage guarantee, and a user-defined criteria; and

in response to a determination that the predetermined criteria is violated, compress the partially overwritten compression group.

8. The system of claim 7, wherein the instructions to update references comprise instructions executable by the one or more processors to cause the system to add an indirection reference to an indirection block of the file, wherein the indirection block includes a first reference to the data blocks of compression group and the added reference references the uncompressed data.

9. The system of claim 7, wherein the instructions to partially overwrite include instructions executable by the one or more processors to cause the system to:

allocate an extent identifier;

determine an offset into an existing extent associated with the compression group, the offset comprising a difference between a first block number provided in the write request and a second block number for a start of the existing extent, wherein the file block identifier is the first block number;

in response to a determination that the offset is non-zero,

generate a first extent entry having the allocated extent identifier and a reference to a start of the compression group,

generate a second extent entry having a reference to the uncompressed data,

generate a third extent entry having a reference to a remainder portion of the compression group that is not overwritten by the uncompressed data, and

associate the first extent entry, the second extent entry, and the third extent entry with the first block number.

10. The system of claim 7, wherein the instructions further comprise instructions executable by the one or more processors to cause the system to:

determine whether the partially overwritten compression group including the uncompressed data should be compressed;

defer compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed; and

compress the partially overwritten compression group if the partially overwritten compression group should be compressed.

11. The system of claim 10, wherein the instructions to determine whether the partially overwritten compression group should be compressed comprise instruction executable by the one or more processors to cause the system to:

determine whether a snapshot of data is to occur within a predetermined amount of time;

determine that the partially overwritten compression group should be compressed if the snapshot of data is to occur within a predetermined amount of time; and

determine that the partially overwritten compression group should not be compressed if the snapshot of data is not to occur within a predetermined amount of time.

12. The system of claim 10, wherein the instructions to determine whether the partially overwritten compression group including the uncompressed data should be compressed comprise instructions executable by the one or more processors to cause the system to:

determine whether the data in the compression group is cold data;

determine that the partially overwritten compression group should be compressed if the data in the compression group is cold data; and

determine that the partially overwritten compression group should not be compressed if the data in the compression group is not cold data.

13. A non-transitory computer-readable storage medium embodied with executable instructions to:

in response to receipt of a write request that indicates a file block identifier and that indicates data to be written, determine whether the file block identifier resolves to a compression group, the compression group comprising a group of data blocks that is compressed; and

partially overwrite the compression group while the compression group remains compressed in response to a determination that the file block identifier resolves to the compression group, wherein the instructions to partially overwrite comprise instructions to,

store the data of the write request as uncompressed data outside the compression group, and

update references used for resolving file block identifiers of the file to data blocks of the file, wherein the instructions to update comprise instructions to update the references to map the file block identifier to the compression group and to the uncompressed data;

determine whether a predetermined criteria is violated by the partial overwriting of the compression group, wherein the predetermined criteria comprises at least one of a system usage guarantee, a space usage guarantee, and a user-defined criteria; and

in response to a determination that the predetermined criteria is violated, compress the partially overwritten compression group.

14. The non-transitory computer-readable storage medium of claim 13, wherein the instructions to update references comprise instructions to add an indirection reference to an indirection block of the file, wherein the indirection block includes a first reference to the data blocks of the compression group and the added reference references the uncompressed data.

15. The non-transitory computer-readable storage medium of claim 13, wherein the instructions to partially overwrite comprise instructions to:

allocate an extent identifier;

write an extent entry in a first sorted data structure, the extent entry including the allocated extent identifier, a reference to the uncompressed, and a compressed bit set to a predetermined value, wherein the allocated extent identifier is used to determine a location of the extent entry in the first sorted data structure;

select an entry in a second sorted data structure for a file block identifier corresponding to the uncompressed data in the write request; and

overwrite an extent identifier in the selected entry with the allocated extent identifier, wherein the file block identifier is used to determine a location of the selected entry in the second sorted data structure.

16. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions to:

determine whether the partially overwritten compression group including the uncompressed data should be compressed;

defer compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed; and

compress the partially overwritten compression group if the partially overwritten compression group should be compressed.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions to determine whether the partially overwritten compression group should be compressed comprises:

determine whether a snapshot of data is to occur within a predetermined amount of time;

determine that the partially overwritten compression group should be compressed if the snapshot of data is to occur within a predetermined amount of time; and

determine that the partially overwritten compression group should not be compressed if the snapshot of data is not to occur within a predetermined amount of time.