METHOD, APPARATUS AND COMPUTER PROGRAM PRODUCT FOR STORING DATA

• US 20200133583A1
• Filed: 09/20/2019
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Xiao, Pan, Chen, Peilei, Du, Fang, Chen, Xu
• Granted Patent: US 11,429,317 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/3071 Prediction
• CPC Class Codes: H03M 7/6011 Encoder aspects
• CPC Class Codes: H03M 7/6035 Handling of unkown probabilities

Abstract

Techniques involve storing data. In particular, such techniques involve: obtaining first data to be stored; determining whether the first data is able to be compressed in a compression ratio exceeding a predetermined threshold; and storing, based on the determined result, the first data into a storage device. Accordingly, such techniques can execute corresponding processing for data in a predicted compression ratio, so as to store the data into a storage device. In this manner, such techniques can significantly cut down the overheads for processing data while minimizing a storage space required for storing data.

Claims

1. A method of storing data, comprising:

obtaining first data to be stored;

determining whether the first data is able to be compressed in a compression ratio exceeding a predetermined threshold; and

storing, based on the determined result, the first data into a storage device.

2. The method according to claim 1, wherein obtaining the first data comprises:

obtaining second data to be stored;

partitioning the second data into a group of data blocks; and

obtaining, from the group of data blocks, a plurality of consecutive data blocks as the first data.

3. The method according to claim 1, wherein determining whether the first data is able to be compressed in a compression ratio exceeding the predetermined threshold comprises:

determining, using a trained prediction model, whether the first data is able to be compressed in a compression ratio exceeding the predetermined threshold.

4. The method according to claim 1, wherein the first data comprise a plurality of data blocks, and wherein storing the first data comprises:

in response to determining that each of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold, compressing the plurality of compression blocks; and

storing the compressed plurality of data blocks into the storage device.

5. The method according to claim 1, wherein the first data comprise a plurality of data blocks, and wherein storing the first data comprises:

in response to determining that each of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, storing the uncompressed plurality of data blocks into the storage device.

6. The method according to claim 1, wherein the first data comprise a plurality of data blocks, and wherein storing the first data comprises:

in response to determining that a part of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold and another part of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, for each of the plurality of data blocks,

compressing the data block;

determining a compression ratio of the data block;

in response to the compression ratio of the data block exceeding the predetermined threshold, storing the compressed data block into the storage device; and

in response to the compression ratio of the data block being below the predetermined threshold, storing the uncompressed data block into the storage device.

7. A method of training a prediction model, comprising:

obtaining a training dataset for training a prediction model; and

training the prediction model using the training dataset, wherein the prediction model is provided for determining whether data to be stored is able to be compressed in a compression ratio exceeding the predetermined threshold, so that the data to be stored is stored based on the determined result.

8. The method according to claim 7, wherein obtaining the training dataset comprises:

obtaining third data for training the prediction model;

determining whether the third data is able to be compressed in a compression ratio exceeding the predetermined threshold; and

labelling, based on the determined result, the third data as training data in the training dataset.

9. The method according to claim 8, wherein obtaining the third data comprises:

obtaining uncompressed fourth data for training the prediction model;

partitioning the fourth data into a group of data blocks; and

obtaining, from the group of data blocks, a plurality of consecutive data blocks as the third data.

10. The method according to claim 8, wherein the third data comprise a plurality of data blocks, and wherein determining whether the third data is able to be compressed in a compression ratio exceeding the predetermined threshold comprises:

for each of the plurality of data blocks,

compressing the data block;

determining a compression ratio of the data block;

in response to the compression ratio of the data block exceeding the predetermined threshold, determining that the data block is able to be compressed in the compression ratio exceeding the predetermined threshold; and

in response to the compression ratio of the data block being below the predetermined threshold, determining that the data block is unable to be compressed in the compression ratio exceeding the predetermined threshold.

11. The method according to claim 8, wherein the third data comprise a plurality of data blocks, and wherein labelling the third data comprises:

in response to determining that each of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold, labelling the third data using a first label indicative of the determined result.

12. The method according to claim 8, wherein the third data comprise a plurality of data blocks, and wherein labelling the third data comprises:

in response to determining that each of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, labelling the third data using a second label indicative of the determined result.

13. The method according to claim 8, wherein the third data comprise a plurality of data blocks, and wherein labelling the third data comprises:

in response to determining that a part of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold and another part of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, labelling the third data using a third label indicative of the determined result.

14. An apparatus for storing data, comprising:

at least one processing unit;

at least one memory coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the apparatus to execute acts of;

obtaining first data to be stored;

determining whether the first data is able to be compressed in a compression ratio exceeding a predetermined threshold; and

storing, based on the determined result, the first data into a storage device.

15. The apparatus according to claim 14, wherein obtaining the first data comprises:

obtaining second data to be stored;

partitioning the second data into a group of data blocks; and

obtaining, from the group of data blocks, a plurality of consecutive data blocks as the first data.

16. The apparatus according to claim 14, wherein determining whether the first data is able to be compressed in a compression ratio exceeding the predetermined threshold comprises:

determining, using a trained prediction model, whether the first data is able to be compressed in a compression ratio exceeding the predetermined threshold.

17. The apparatus according to claim 14, wherein the first data comprise a plurality of data blocks, and wherein storing the first data comprises:

in response to determining that each of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold, compressing the plurality of compression blocks; and

storing the compressed plurality of data blocks into the storage device.

18. The apparatus according to claim 14, wherein the first data comprise a plurality of data blocks, and wherein storing the first data comprises:

in response to determining that each of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, storing the uncompressed plurality of uncompressed data blocks into the storage device.

19. The apparatus according to claim 14, wherein the first data comprise a plurality of data blocks, and wherein storing the first data comprises:

in response to determining that a part of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold and another part of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, for each of the plurality of data blocks,

compressing the data block;

determining a compression ratio of the data block;

in response to the compression ratio of the data block exceeding the predetermined threshold, storing the compressed data block into the storage device; and

in response to the compression ratio of the data block being below the predetermined threshold, storing the uncompressed data block into the storage device.

20. An apparatus for training a prediction model, comprises:

at least one processing unit;

at least one memory coupled to the at least one processing unit and storing instructions executed by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the apparatus to execute acts of;

obtaining a training dataset for training a prediction model; and

training the prediction model using the training dataset, wherein the prediction model is provided for determining whether data to be stored is able to be compressed in a compression ratio exceeding the predetermined threshold, so that the data to be stored is stored based on the determined result.

21. The apparatus according to claim 20, wherein obtaining the training dataset comprises:

obtaining third data for training the prediction model;

determining whether the third data is able to be compressed in a compression ratio exceeding the predetermined threshold; and

labelling, based on the determined result, the third data as training data in the training dataset.

22. The apparatus according to claim 21, wherein obtaining the third data comprises:

obtaining uncompressed fourth data for training the prediction model;

partitioning the fourth data into a group of data blocks; and

obtaining, from the group of data blocks, a plurality of consecutive data blocks as the third data.

23. The apparatus according to claim 21, wherein the third data comprise a plurality of data blocks, and wherein determining whether the third data is able to be compressed in a compression ratio exceeding the predetermined threshold comprises:

for each of the plurality of data blocks,

compressing the data block;

determining a compression ratio of the data block;

in response to the compression ratio of the data block exceeding the predetermined threshold, determining that the data block is able to be compressed in the compression ratio exceeding the predetermined threshold; and

in response to the compression ratio of the data block being below the predetermined threshold, determining that the data block is unable to be compressed in the compression ratio exceeding the predetermined threshold.

24. The apparatus according to claim 21, wherein the third data comprise a plurality of data blocks, and wherein labelling the third data comprises:

in response to determining that each of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold, labelling the third data using a first label indicative of the determined result.

25. The apparatus according to claim 21, wherein the third data comprise a plurality of data blocks, and wherein labelling the third data comprises:

in response to determining that each of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, labelling the third data using a second label indicative of the determined result.

26. The apparatus according to claim 21, wherein the third data comprise a plurality of data blocks, and wherein labelling the third data comprises:

in response to determining that a part of the plurality of data blocks is able to be compressed in a compression ratio exceeding the predetermined threshold and another part of the plurality of data blocks is unable to be compressed in a compression ratio exceeding the predetermined threshold, labelling the third data using a third label indicative of the determined result.

27. A computer program product having a non-transitory computer readable medium which stores a set of instructions to store data;

the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of;

obtaining first data to be stored;

determining whether the first data is able to be compressed in a compression ratio exceeding a predetermined threshold; and

storing, based on the determined result, the first data into a storage device.

28. A computer program product having a non-transitory computer readable medium which stores a set of instructions to train a prediction model;

the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of;

obtaining a training dataset for training a prediction model; and

training the prediction model using the training dataset, wherein the prediction model is provided for determining whether data to be stored is able to be compressed in a compression ratio exceeding the predetermined threshold, so that the data to be stored is stored based on the determined result.