Data integrity and acceleration in compressed storage environments in combination with software defined native RAID

• US 9,846,538 B2
• Filed: 12/07/2015
• Issued: 12/19/2017
• Est. Priority Date: 12/07/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Rueger, Erik, Russell, Lance W., Schmitt, Christof, Snellgrove, Thomas A.
• Primary Examiner(s): BRYAN, JASON B
• Application Number:
  US14/961,196
• Publication Number:
  US 20170160948A1
• Time in Patent Office: 743 Days
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A mechanism is provided in a data processing system for data integrity and acceleration in compressed storage environments in combination with software defined native redundant array of independent disks (RAID). The mechanism receives a data portion to write to storage and divides the data portion into a plurality of chunks. The mechanism performs real-time compression on a given chunk within the plurality of chunks and determines a compression rate for the given chunk. The mechanism determines a RAID level for the given chunk based on a value of the compression rate and stores the given chunk based on the determined RAID level.

Claims

1. A method, in a data processing system, for data integrity and acceleration in compressed storage environments in combination with software defined native redundant array of independent disks (RAID), the method comprising:

receiving a data portion to write to storage;

dividing the data portion into a plurality of chunks;

performing, by a real-time compression engine, real-time compression on a given chunk within the plurality of chunks to form a compressed chunk;

receiving, by a data handling module, the compressed chunk and a compression rate for the given chunk from the real-time compression engine;

determining, by the data handling module, a RAID level for the given chunk based on a value of the compression rate; and

storing, by the data handling module, the compressed chunk based on the determined RAID level.

2. The method of claim 1, wherein determining the RAID level for the given chunk comprises comparing the value of the compression rate to at least one threshold, wherein the at least one threshold are mapped to corresponding RAID levels.

3. The method of claim 2, wherein values of the compression rate less than a first threshold are mapped to RAID 5, wherein values of the compression rate greater than the first threshold and less than a second threshold are mapped to RAID 6, wherein values of the compression rate greater than the second threshold and less than a third threshold are mapped to RAID 10, and wherein values of the compression rate greater than the third threshold are mapped to RAID 1.

4. The method of claim 2, wherein storing the given chunk comprises migrating the given chunk from a previous storage pool to a new storage pool responsive to the value of the compression rate crossing a threshold within the at least one threshold.

5. The method of claim 1, further comprising determining a performance level for the given chunk based on the value of the compression rate, wherein storing the given chunk comprises storing the given chunk based on the determined RAID level and the determined performance level.

6. The method of claim 5, wherein determining the performance level for the given chunk comprises comparing the value of the compression rate to at least one threshold, wherein the at least one threshold are mapped to corresponding RAID levels.

7. The method of claim 6, wherein values of the compression rate less than a first threshold are mapped to a low performance pool of storage, wherein values of the compression rate greater than the first threshold and less than a second threshold are mapped to a medium performance pool of storage, and wherein values of the compression rate greater than the second threshold are mapped to a high performance pool of storage.

8. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

receive a data portion to write to storage;

divide the data portion into a plurality of chunks;

perform, by a real-time compression engine, real-time compression on a given chunk within the plurality of chunks;

receive, by a data handling module, the compressed chunk and a compression rate for the given chunk from the real-time compression engine;

determine, by the data handling module, a RAID level for the given chunk based on a value of the compression rate; and

store the given chunk based on the determined RAID level.

9. The computer program product of claim 8, wherein determining the RAID level for the given chunk comprises comparing the value of the compression rate to at least one threshold, wherein the at least one threshold are mapped to corresponding RAID levels.

10. The computer program product of claim 9, wherein values of the compression rate less than a first threshold are mapped to RAID 5, wherein values of the compression rate greater than the first threshold and less than a second threshold are mapped to RAID 6, wherein values of the compression rate greater than the second threshold and less than a third threshold are mapped to RAID 10, and wherein values of the compression rate greater than the third threshold are mapped to RAID 1.

11. The computer program product of claim 9, wherein storing the given chunk comprises migrating the given chunk from a previous storage pool to a new storage pool responsive to the value of the compression rate crossing a threshold within the at least one threshold.

12. The computer program product of claim 8, further comprising determining a performance level for the given chunk based on the value of the compression rate, wherein storing the given chunk comprises storing the given chunk based on the determined RAID level and the determined performance level.

13. The computer program product of claim 12, wherein determining the performance level for the given chunk comprises comparing the value of the compression rate to at least one threshold, wherein the at least one threshold are mapped to corresponding RAID levels.

14. The computer program product of claim 13, wherein values of the compression rate less than a first threshold are mapped to a low performance pool of storage, wherein values of the compression rate greater than the first threshold and less than a second threshold are mapped to a medium performance pool of storage, and wherein values of the compression rate greater than the second threshold are mapped to a high performance pool of storage.

15. An apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

receive a data portion to write to storage;

divide the data portion into a plurality of chunks;

perform, by a real-time compression engine, real-time compression on a given chunk within the plurality of chunks;

receive, by a data handling module, the compressed chunk and a compression rate for the given chunk from the real-time compression engine;

determine, by the data handling module, a RAID level for the given chunk based on a value of the compression rate; and

store, by the data handling module, the given chunk based on the determined RAID level.

16. The apparatus of claim 15, wherein determining the RAID level for the given chunk comprises comparing the value of the compression rate to at least one threshold, wherein the at least one threshold are mapped to corresponding RAID levels.

17. The apparatus of claim 16, wherein values of the compression rate less than a first threshold are mapped to RAID 5, wherein values of the compression rate greater than the first threshold and less than a second threshold are mapped to RAID 6, wherein values of the compression rate greater than the second threshold and less than a third threshold are mapped to RAID 10, and wherein values of the compression rate greater than the third threshold are mapped to RAID 1.

18. The apparatus of claim 15, further comprising determining a performance level for the given chunk based on the value of the compression rate, wherein storing the given chunk comprises storing the given chunk based on the determined RAID level and the determined performance level.

19. The apparatus of claim 18, wherein determining the performance level for the given chunk comprises comparing the value of the compression rate to at least one threshold, wherein the at least one threshold are mapped to corresponding RAID levels.

20. The apparatus of claim 19, wherein values of the compression rate less than a first threshold are mapped to a low performance pool of storage, wherein values of the compression rate greater than the first threshold and less than a second threshold are mapped to a medium performance pool of storage, and wherein values of the compression rate greater than the second threshold are mapped to a high performance pool of storage.