Overwriting part of compressed data without decompressing on-disk compressed data

• US 8,745,338 B1
• Filed: 05/02/2011
• Issued: 06/03/2014
• Est. Priority Date: 05/02/2011
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Yadav, Sandeep, Faith, Rickard E., Periyagaram, Subramaniam, Lewis, Blake, Prakash, Ashish
• Primary Examiner(s): Bradley, Matthew
• Assistant Examiner(s): RANKIN, CANDICE A
• Application Number:
  US13/099,283
• Time in Patent Office: 1,128 Days
• US Class Current: 711/154
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 2211/1014 Compression, i.e. RAID systems with parity using compression techniques
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Overwriting part of compressed data without decompressing on-disk compressed data is implemented by receiving a write request for a block of data in a compression group from a client, wherein the compression group comprises a group of data blocks that is compressed, wherein the block of data is uncompressed. The storage server partially overwrites the compression group, wherein the compression group remains compressed while the partial overwriting is performed. The storage server determines whether the partially overwritten compression group including the uncompressed block of data should be compressed. The storage server defers compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed. The storage server compresses the partially overwritten compression group if the partially overwritten compression group should be compressed.

Claims

1. A computerized method comprising:

receiving, by a storage server from a client, a write request for a block of data in a compression group pointed to by an indirection block, the compression group comprising a group of data blocks that is compressed;

partially overwriting, by the storage server, the compression group while the compression group remains compressed, wherein the partial overwriting comprises,

storing the block of data of the write request as an uncompressed block outside the compression group, and

generating an indirection reference in the indirection block to point to the uncompressed block as well as the compression group;

determining at least one of whether one or more predetermined criteria are violated by the partial overwriting of the compression group if the one or more predetermined criteria exist for the storage server and whether a snapshot of data is to occur within a predetermined amount of time, wherein the one or more predetermined criteria comprise at least one of a system usage guarantee, a space usage guarantee, and a user-defined criteria;

determining, by the storage server, whether the partially overwritten compression group including the uncompressed block of data should be compressed, wherein said determining whether the partially overwritten compression group including the uncompressed block of data should be compressed comprises,

determining that the partially overwritten compression group should be compressed if the one or more predetermined criteria exist for the storage server and are violated;

determining that the partially overwritten compression group should not be compressed if the one or more predetermined criteria are not violated;

determining that the partially overwritten compression group should be compressed if the snapshot of data is to occur within a predetermined amount of time,

determining that the partially overwritten compression group should not be compressed if the snapshot of data is not to occur within a predetermined amount of time;

deferring, by the storage server, compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed; and

compressing the partially overwritten compression group if the partially overwritten compression group should be compressed.

2. The computerized method of claim 1, further comprising:

creating the compression group, wherein creating the compression group comprises;

grouping, by the storage server, a predetermined number of file block numbers (FBNs) into the compression group;

compressing, by the storage server, data in physical blocks associated with the FBNs;

writing, by the storage server, a first entry in a first sorted data structure, the first entry comprising an extent identifier and a reference to an extent in the storage server, wherein the extent identifier is used to determine a location of the first entry in the first sorted data structure; and

writing, by the storage server, an entry in a second sorted data structure for each of the predetermined number of FBNs, each entry including an FBN and the extent identifier, wherein the FBN is used to determine a location of the entry in the second sorted data structure.

3. The computerized method of claim 1, wherein partially overwriting comprises:

allocating an extent identifier;

writing an extent entry in a first sorted data structure, the extent entry including the allocated extent identifier, a reference to the block of data in the write request, and a compressed bit set to a predetermined value, wherein the allocated extent identifier is used to determine a location of the extent entry in the first sorted data structure;

selecting an entry in a second sorted data structure for a FBN corresponding to the block of data in the write request; and

overwriting an extent identifier in the selected entry with the allocated extent identifier, wherein the FBN is used to determine a location of the determined entry in the second sorted data structure.

4. The computerized method of claim 1, wherein the determining whether the partially overwritten compression group should be compressed comprises:

determining whether the data in the compression group is cold data;

determining that the partially overwritten compression group should be compressed if the data in the compression group is cold data, and

determining that the partially overwritten compression group should not be compressed if the data in the compression group is not cold data.

5. A storage server comprising:

a memory;

a processor coupled to the memory through a bus, wherein the processor executes instructions that cause the storage server to,

receive a write request for a block of data in a compression group pointed to by an indirection block, the compression group comprising a group of data blocks that is compressed;

partially overwrite the compression group while, the compression group remains compressed, wherein the partial overwriting comprises,

storing the block of data of the write request as an uncompressed block outside the compression group, and

generating an indirection reference in the indirection block to point to the uncompressed block as well as the compression group;

determine at least one of whether one or more predetermined criteria are violated by the partial overwriting of the compression group if the one or more predetermined criteria exist for the storage server and whether a snapshot of data is to occur within a predetermined amount of time, wherein the one or more predetermined criteria comprise at least one of a system usage guarantee, a space usage guarantee, and a user-defined criteria;

determine whether the partially overwritten compression group including the uncompressed block of data should be compressed, wherein the program instructions executable by the processor to cause the storage server to whether the partially overwritten compression group including the uncompressed block of data should be compressed comprises program instructions executable by the processor to cause the storage server to,

determine that the partially overwritten compression group should be compressed if the one or more predetermined criteria exist for the storage server and are violated;

determine that the partially overwritten compression group should not be compressed if the one or more predetermined criteria are not violated;

determine that the partially overwritten compression group should be compressed if the snapshot of data is to occur within a predetermined amount of time,

determine that the partially overwritten compression group should not be compressed if the snapshot of data is not to occur within a predetermined amount of time;

defer compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed; and

compress the partially overwritten compression group if the partially overwritten compression group should be compressed.

6. The storage server of claim 5, wherein the processor further executes instructions that cause the storage server to:

create the compression group, wherein to create the compression group the processor further executes instructions that cause the processor to

group a predetermined number of file block numbers (FBNs) into the compression group;

compress data in physical blocks associated with the FBNs;

write a first entry in a first sorted data structure, the first entry comprising an extent identifier and a reference to an extent in the storage server, wherein the extent identifier is used to determine a location of the first entry in the first sorted data structure; and

write an entry in a second sorted data structure for each of the predetermined number of FBNs, each entry including an FBN and the extent identifier, wherein the FBN is used to determine a location of the entry in the second sorted data structure.

7. The storage server of claim 5, wherein to partially overwrite, the processor further executes instructions that cause the storage server to:

allocate an extent identifier;

write an extent entry in a first sorted data structure, the extent entry including the allocated extent identifier, a reference to the block of data in the write request, and a compressed bit set to a negative value, wherein the allocated extent identifier is used to determine a location of the extent entry in the first sorted data structure;

determine an entry in a second sorted data structure for a FBN corresponding to the block of data in the write request; and

overwrite an extent identifier in the determined entry with the allocated extent identifier, wherein the FBN is used to determine a location of the determined entry in the second sorted data structure.

8. The storage server of claim 5, wherein to determine whether the partially overwritten compression group should be compressed, the processor further executes instructions that cause the storage server to:

determine whether the data in the compression group is cold data;

determine that the partially overwritten compression group should be compressed if the data in the compression group is cold data, and

determine that the partially overwritten compression group should not be compressed if the data in the compression group is not cold data.

9. A non-transitory computer-readable storage medium embodied with executable instructions that cause a processor to perform operations comprising:

receiving a write request for a block of data in a compression group pointed to by an indirection block, the compression group comprising a group of data blocks that is compressed;

partially overwriting the compression group while the compression group remains compressed, wherein the partial overwriting comprises,

storing the block of data of the write request as an uncompressed block outside the compression group, and

generating an indirection reference in the indirection block to point to the uncompressed block as well as the compression group;

determining at least one of whether one or more predetermined criteria are violated by the partial overwriting of the compression group if the one or more predetermined criteria exist for the storage server and whether a snapshot of data is to occur within a predetermined amount of time, wherein the one or more predetermined criteria comprise at least one of a system usage guarantee, a space usage guarantee, and a user-defined criteria

determining whether the partially overwritten compression group including the uncompressed block of data should be compressed, wherein said determining whether the partially overwritten compression group including the uncompressed block of data should be compressed comprises,

determining that the partially overwritten compression group should be compressed if the one or more predetermined criteria exist for the storage server and are violated;

determining that the partially overwritten compression group should not be compressed if the one or more predetermined criteria are not violated;

determining that the partially overwritten compression group should be compressed if the snapshot of data is to occur within a predetermined amount of time,

determining that the partially overwritten compression group should not be compressed if the snapshot of data is not to occur within a predetermined amount of time;

deferring compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed; and

compressing the partially overwritten compression group if the partially overwritten compression group should be compressed.

10. The non-transitory computer-readable storage medium of claim 9, further comprising program instructions that, when executed by a processor, cause the processor to perform comprising:

creating the compression group, wherein creating the compression group comprises

creating the compression group, wherein creating the compression group comprises;

grouping, by the storage server, a predetermined number of file block numbers (FBNs) into the compression group;

compressing, by the storage server, data in physical blocks associated with the FBNs;

writing, by the storage server, a first entry in a first sorted data structure, the first entry comprising an extent identifier and a reference to an extent in the storage server, wherein the extent identifier is used to determine a location of the first entry in the first sorted data structure; and

writing, by the storage server, an entry in a second sorted data structure for each of the predetermined number of FBNs, each entry including an FBN and the extent identifier, wherein the FBN is used to determine a location of the entry in the second sorted data structure.

11. The non-transitory computer-readable storage medium of claim 9, wherein the program instructions that, when executed by a processor, cause the processor to perform the operation of partially overwriting comprises program instructions that, when executed by the processor, cause the processor to perform operations comprising:

allocating an extent identifier;

writing an extent entry in a first sorted data structure, the extent entry including the allocated extent identifier, a reference to the block of data in the write request, and a compressed bit set to a predetermined value, wherein the allocated extent identifier is used to determine a location of the extent entry in the first sorted data structure;

selecting an entry in a second sorted data structure for a FBN corresponding to the block of data in the write request; and

overwriting an extent identifier in the selected entry with the allocated extent identifier, wherein the FBN is used to determine a location of the determined entry in the second sorted data structure.

12. The non-transitory computer-readable storage medium of claim 9, wherein the program instructions that, when executed by a processor, cause the processor to perform the operation of determining whether the partially overwritten compression group should be compressed comprises program instructions that, when executed by the processor, cause the processor to perform operations comprising:

determining whether the data in the compression group is cold data;

determining that the partially overwritten compression group should be compressed if the data in the compression group is cold data, and

determining that the partially overwritten compression group should not be compressed if the data in the compression group is not cold data.