Data transformation during recycling

• US 9,400,609 B1
• Filed: 11/04/2015
• Issued: 07/26/2016
• Est. Priority Date: 11/04/2015
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Randall, Charles, Wright, David D., Xu, Michael
• Primary Examiner(s): Rossiter, Sean D
• Application Number:
  US14/932,063
• Time in Patent Office: 265 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

Disclosed are systems, computer-readable mediums, and methods for transforming data in a file system. As part of a recycling process, a determination is made that transformations should be attempted. A data block is determined to be in use by at least one user of the storage system. If a transformation should be attempted on the data block is determined. Parameters associated with the performance of the file system can be used in this determination. A type of transformation to be done is determined. The data block is transformed based upon the selected transformation. The transformed data block is written to the storage system. As part of the recycling process, the transformation requires no additional input/output requests.

Claims

1. A storage system comprising:

one or more electronic processors configured to;

determine that transformations of data should be attempted;

determine a data block is in use by at least one user of the storage system;

determine a transformation of the data block should be done based upon the determination that the data block is in use by at least one user of the storage system;

determine a transformation to be done to the data block based upon the determination that the transformation of the data block should be done;

transform the data block based upon the determined transformation;

write the transformed data block to the storage system; and

recycle, as part of a recycle operation, a portion of storage space to free up space available for data writing, wherein the transformation is integrated into the recycle operation, wherein the transformation requires no additional input/output requests from the recycle operation, and wherein the write the transformed data block to the storage system is part of the recycle operation.

2. The storage system of claim 1, wherein the storage system uses a log-structured file system.

3. The storage system of claim 1, wherein the one or more electronic processors are further configured to:

receive a request from one user of the storage system to store data in the storage system, wherein the request comprises the data, and wherein the data comprises the data block;

compress the data block using a first compression algorithm; and

write the data block to the storage system prior to the recycle operation.

4. The storage system of claim 3, wherein the determined transformation is compression using a second, different compression algorithm, and wherein to transform the data block the one or more electronic processors are configured to:

uncompress the data block using the first compression algorithm; and

compress the data block using the second compression algorithm.

5. The storage system of claim 4, wherein to transform the data block the one or more electronic processors are configured to:

compare the size of the compressed data block using the second compression algorithm with the size of the compressed data block using the first compression algorithm; and

determine the smallest compressed data block based upon the comparison, wherein the smallest compressed data block is written to the storage system as part of the recycling.

6. The storage system of claim 3, wherein the one or more electronic processors are further configured to:

determine that a usage level of storage system resources for decompression of data is above a predetermined threshold;

wherein to determine a transformation to be done to the data block, the one or more electronic processors are configured to;

determine a compression algorithm used to compress the data block; and

determine a second compression algorithm that requires less resources than the compression algorithm used to compress the data block;

wherein to transform the data block the one or more electronic processors are configured to;

decompress the data block using the compression algorithm used to compress the data block; and

compress the data block using the second compression algorithm.

7. The storage system of claim 3, wherein the one or more electronic processors are further configured to:

determine that a usage level of storage system resources for decompression of data is above a predetermined threshold, wherein the determined transformation is uncompression;

wherein to transform the data block the one or more electronic processors are configured to decompress the data block using a compression algorithm used to compress the data block, wherein the decompressed data block is written to the storage system as part of the recycling.

8. The storage system of claim 1, wherein the one or more electronic processors are further configured to:

receive a request from one user of the storage system to store data in the storage system, wherein the request comprises the data, and wherein the data comprises the data block;

determine an initial compression should not occur due to parameters of the storage system; and

write the data block to the storage system prior to the recycle operation.

9. The storage system of claim 8, wherein the determined transformation is compression using a compression algorithm, and wherein to transform the data block the one or more electronic processors are configured to compress the data block using the compression algorithm.

10. The storage system of claim 1, wherein the one or more electronic processors are further configured to:

encrypt the data block using a first encryption key prior to the write of the data block to storage, wherein the determined transformation is re-encryption; and

wherein to transform the data block the one or more electronic processors are configured to;

decrypt the data block; and

encrypt the data block using a second, different, encryption key.

11. The storage system of claim 1, wherein the one or more electronic processors are further configured to:

receive a request from one user of the storage system to store data in the storage system, wherein the request comprises the data, and wherein the data comprises the data block;

encrypt the data block using a first encryption algorithm;

write the data block to the storage system prior to the recycle operation;

wherein to determine a transformation to be done to the data block, the one or more electronic processors are configured to;

decrypt the data block based upon the first encryption algorithm.

12. The storage system of claim 1, wherein the one or more electronic processors are further configured to:

determine a second transformation to be done to the data block;

transform the transformed data block based upon the second transformation before the transformed data block is written to the storage system.

13. A method comprising:

determining, using one or more electronic processors, that transformations of data should be attempted;

determining, using the one or more electronic processors, a data block is in use by at least one user of the storage system;

determining, using the one or more electronic processors, that a transformation of the data block should be done based upon the determination that the data block is in use by at least one user of the storage system;

determining, using the one or more electronic processors, a transformation to be done to the data block based upon the determination that the transformation of the data block should be done;

transforming, using the one or more electronic processors, the data block based upon the determined transformation;

writing, using the one or more electronic processors, the transformed data block to the storage system; and

performing a recycling process to free up a portion of storage space available for data writing, wherein the transformation is integrated into the recycling process, wherein the transformation requires no additional input/output requests from the recycling process, and wherein the write the data block to the storage system is part of the recycling process.

14. The method of claim 13, wherein the storage system uses a log-structured file system.

15. The method of claim 13, further comprising:

receiving a request from one user of the storage system to store data in the storage system, wherein the request comprises the data, and wherein the data comprises the data block;

determining an initial compression should not occur due to parameters of the storage system; and

writing the data block to the storage system prior to the recycle operation.

16. The method system of claim 13, further comprising:

encrypting the data block using a first encryption key prior to the write of the data block to storage, wherein the determined transformation is re-encryption; and

wherein transforming the data block comprises;

decrypting the data block; and

encrypting the data block using a second, different, encryption key.

17. A non-transitory computer-readable medium having instructions stored thereon, the instructions comprising:

instructions to determine that transformations of data should be attempted;

instructions to determine that a data block is in use by at least one user of the storage system;

instructions to determine that a transformation of the data block should be done based upon the determination that the data block is in use by at least one user of the storage system;

instructions to determine a transformation to be done to the data block based upon the determination that the transformation of the data block should be done;

instructions to transform the data block based upon the determined transformation;

instructions to write the transformed data block to the storage system;

instructions to perform a recycling process to free up a portion of storage space available for writer, wherein the transformation is integrated into the recycling process, wherein the transformation requires no additional input/output requests from the recycling process, and wherein the write the data block to the storage system is part of the recycling process.

18. The non-transitory computer-readable medium of claim 17, wherein the storage system uses a log-structured file system.

19. The non-transitory computer-readable medium of claim 17, wherein the instructions further comprise:

instructions to receive a request from one user of the storage system to store data in the storage system, wherein the request comprises the data, and wherein the data comprises the data block;

instructions to determine an initial compression should not occur due to parameters of the storage system; and

instructions to write the data block to the storage system prior to the recycle operation.

20. The non-transitory computer-readable medium of claim 17, wherein the instructions further comprise:

instructions to encrypt the data block using a first encryption key prior to the write of the data block to storage, wherein the determined transformation is re-encryption; and

wherein the instructions to transform the data block comprises;

instructions to decrypt the data block; and

instructions to encrypt the data block using a second, different, encryption key.