Dynamic data compression

• US 10,776,026 B2
• Filed: 01/23/2019
• Issued: 09/15/2020
• Est. Priority Date: 12/02/2016
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Cheah, ChernYih, Tatiparthi, Kiran, Bhattacharyya, Manosiz, Arora, Varun Kumar
• Primary Examiner(s): Myers, Paul R.
• Application Number:
  US16/255,242
• Publication Number:
  US 20190171382A1
• Time in Patent Office: 601 Days
• Field of Search: 710 68
• CPC Class Codes: G06F 13/20 for access to input/output bus
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.

Claims

1. A method, comprising:

receiving a request to write data from an application;

partitioning the data into a plurality of data units;

processing the plurality of data units to execute the request to write data from the application, wherein each data unit of the plurality of data units is processed by;

applying a compression rule to a respective data unit of the plurality of data units to determine a corresponding compression parameter for the respective data unit; and

performing, in accordance with the corresponding compression parameter for the respective data, an in-line data compression operation on the respective data unit, wherein the in-line data compression operation is executed in-line with writing the respective data unit in response to the request to write data.

2. The method of claim 1, wherein the plurality of data units are processed before returning a status pertaining to the execution of the request to write data from the application.

3. The method of claim 1, wherein processing the respective data units further comprises performing a compression efficiency test over the data unit to determine a data unit attribute associated with the request to write data.

4. The method of claim 1, wherein a first data unit from the data is compressed using a first compression parameter and a second data unit from the data is compressed using a second compression parameter different from the first compression parameter.

5. The method of claim 1, wherein processing the respective data units further comprises updating a set of metadata to describe the corresponding compression parameter for the respective data unit and metadata for a first data unit from the data is updated to reflect an attribute of compression of the first data unit and metadata for a second data unit from the data is updated to reflect an attribute of compression of the second data unit different from the attribute of compression of the first data unit.

6. The method of claim 1, wherein a first data unit from the data is compressed using a first algorithm and a second data unit from the data is compressed using a second algorithm different from the first algorithm.

7. The method of claim 1, wherein a first data unit from the data is compressed at a first level and a second data unit from the data is compressed using at second level different from the first level.

8. The method of claim 1, wherein the corresponding compression parameter characterize at least one of, a storage device, a compression level, a compression type, a compression algorithm, a storage allocation cushion, or a compression delay.

9. The method of claim 1, wherein the data comprises at least one of, an extent, a slice, a block, an area, or a region.

10. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts comprising:

receiving a request to write data from an application;

partitioning the data into a plurality of data units;

processing the plurality of data units to execute the request to write data from the application, wherein each data unit of the plurality of data units is processed by;

applying a compression rule to a respective data unit of the plurality of data units to determine a corresponding compression parameter for the respective data unit; and

performing, in accordance with the corresponding compression parameter for the respective data, an in-line data compression operation on the respective data unit, wherein the in-line data compression operation is executed in-line with writing the respective data unit in response to the request to write data.

11. The computer readable medium of claim 10, wherein the plurality of data units are processed before returning a status pertaining to the execution of the request to write data from the application.

12. The computer readable medium of claim 10, wherein processing the respective data units further comprises performing a compression efficiency test over the data unit to determine a data unit attribute associated with the request to write data.

13. The computer readable medium of claim 10, wherein a first data unit from the data is compressed using a first compression parameter and a second data unit from the data is compressed using a second compression parameter different from the first compression parameter.

14. The computer readable medium of claim 10, wherein processing the respective data units further comprises updating a set of metadata to describe the corresponding compression parameter for the respective data unit and metadata for a first data unit from the data is updated to reflect an attribute of compression of the first data unit and metadata for a second data unit from the data is updated to reflect an attribute of compression of the second data unit different from the attribute of compression of the first data unit.

15. The computer readable medium of claim 10, wherein a first data unit from the data is compressed using a first algorithm and a second data unit from the data is compressed using a second algorithm different from the first algorithm.

16. The computer readable medium of claim 10, wherein a first data unit from the data is compressed at a first level and a second data unit from the data is compressed using at second level different from the first level.

17. The computer readable medium of claim 10, wherein the corresponding compression parameter characterize at least one of, a storage device, a compression level, a compression type, a compression algorithm, a storage allocation cushion, or a compression delay.

18. The computer readable medium of claim 10, wherein the data comprises at least one of, an extent, a slice, a block, an area, or a region.

19. A system, comprising:

a processor to execute a set of instructions;

a memory to hold the set of instructions, in which the set of instructions comprises instructions to perform;

receiving a request to write data from an application;

partitioning the data into a plurality of data units;

processing the plurality of data units to execute the request to write, wherein each data unit of the plurality of data units is processed by;

applying a compression rule to a respective data unit of the plurality of data units to determine a corresponding compression parameter for the respective data unit; and

performing, in accordance with the corresponding compression parameter for the respective data, an in-line data compression operation on the respective data unit, wherein the in-line data compression operation is executed in-line with writing the respective data unit in response to the request to write data.

20. The system of claim 19, wherein the plurality of data units are processed before returning a status pertaining to the execution of the request to write data from the application.

21. The system of claim 19, wherein processing the respective data units further comprises performing a compression efficiency test over the data unit to determine a data unit attribute associated with the request to write data.

22. The system of claim 19, wherein a first data unit from the data is compressed using a first compression parameter and a second data unit from the data is compressed using a second compression parameter different from the first compression parameter.

23. The system of claim 19, wherein processing the respective data units further comprises updating a set of metadata to describe the corresponding compression parameter for the respective data unit and metadata for a first data unit from the data is updated to reflect an attribute of compression of the first data unit and metadata for a second data unit from the data is updated to reflect an attribute of compression of the second data unit different from the attribute of compression of the first data unit.

24. The system of claim 19, wherein a first data unit from the data is compressed using a first algorithm and a second data unit from the data is compressed using a second algorithm different from the first algorithm.

25. The system of claim 19, wherein a first data unit from the data is compressed at a first level and a second data unit from the data is compressed using at second level different from the first level.

26. The system of claim 19, wherein the corresponding compression parameter characterize at least one of, a storage device, a compression level, a compression type, a compression algorithm, a storage allocation cushion, or a compression delay.

27. The system of claim 19, wherein the data comprises at least one of, an extent, a slice, a block, an area, or a region.