Decision-based data compression by means of deep learning technologies

• US 10,276,134 B2
• Filed: 03/22/2017
• Issued: 04/30/2019
• Est. Priority Date: 03/22/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Diederich, Michael, Doerk, Thomas, Muehge, Thorsten, Rueger, Erik
• Primary Examiner(s): Buttram, Todd
• Application Number:
  US15/465,850
• Publication Number:
  US 20180277068A1
• Time in Patent Office: 769 Days
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 5/39 Control of the bit-mapped memory

Abstract

Data may be handled based on compressibility (i.e., whether the data may be further compressed or is not further compressible). A supervised learning model may be trained using a set of known further compressible data and a set of known non-compressible data. Using these data sets, the model may generate weighting factors and bias for the particular data sets. The trained model may then be used to evaluate a set of unclassified data.

Claims

1. A method for creating a model for handling data based on compressibility of the data, the method comprising:

identifying, by a computer system, a first training data stream input, the first data stream input comprising a sample of fully compressed data;

identifying, by the computer system, a second training data stream input, the second data stream input comprising a sample of compressible data;

evaluating, using an artificial neural network (ANN), the first training data stream input and the second training data stream input;

tagging, as fully compressed, the first training data stream output; and

tagging, as compressible, the second training data stream output.

2. The method of claim 1, further comprising:

identifying, following the evaluating of the first and second training data stream inputs and by the computer system, a digital data stream to be compressed by the computer system;

evaluating, using the ANN, the digital data stream to be compressed by the computer system;

determining, based on the evaluating, the data in the digital data stream is not further compressible;

storing the fully compressed digital data in the digital data stream.

3. The method of claim 1, further comprising:

identifying, following the evaluating of the first and second training data stream inputs and by the computer system, a digital data stream to be compressed by the computer system;

evaluating, using the ANN, the digital data stream to be compressed by the computer system;

determining, based on the evaluating, a subset of data in the digital data stream is further compressible data; and

transmitting, to a data compressor module, the subset of data.

4. The method of claim 1, further comprising:

identifying, following the evaluating of the first and second training data stream inputs and by the computer system, digital data to be compressed by the computer system;

evaluating, using the ANN, the digital data to be compressed;

determining, based on the evaluating, a subset of the digital data is fully compressed;

transmitting, to a data compressor module and for compression, the subset of fully compressed data;

determining, based on the fully compressed data from the data compressor module, data compression performance; and

refining, based on the data compression performance, the ANN evaluating.

5. The method of claim 1, wherein the evaluating using an ANN comprises:

determining a set of weighting factors describing the ANN;

applying, to the first training data stream input and the second training data stream input, the set of weighting factors; and

generating, by the applying, the first training data stream output and the second training data stream output.

6. A computer system for handling data based on compressibility, the system comprising a processor configured to perform a method comprising:

identifying a first training data stream input, the first data stream input comprising a sample of fully compressed data;

identifying a second training data stream input, the second data stream input comprising a sample of compressible data;

evaluating, using an artificial neural network (ANN), the first training data stream input and the second training data stream input;

tagging, as fully compressed, the first training data stream output; and

tagging, as compressible, the second training data stream output.

7. The computer system of claim 6, wherein the method performed by the processor further comprises:

identifying, following the evaluating of the first and second training data stream inputs, a digital data stream to be compressed by a data compressor module;

evaluating, using the ANN, the digital data stream to be compressed by the data compressor module;

determining, based on the evaluating, the data in the digital data stream is not further compressible; and

storing the fully compressed digital data in the digital data stream.

8. The computer system of claim 6, wherein the method performed by the processor further comprises:

identifying, following the evaluating of the first and second training data stream inputs, a digital data stream to be compressed by a data compressor module;

evaluating, using the ANN, the digital data stream to be compressed by the data compressor module;

determining, based on the evaluating, a subset of data in the digital data stream is further compressible data; and

transmitting, to the data compressor module, the subset of data.

9. The computer system of claim 6, wherein the method performed by the processor further comprises:

identifying, following the evaluating of the first and second training data stream inputs, digital data to be compressed by a data compressor module;

evaluating, using the ANN, the digital data to be compressed;

determining, based on the evaluating, a subset of the digital data is fully compressed;

transmitting, to the data compressor module and for compression, the subset of fully compressed data;

determining, based on the fully compressed data from the data compressor module, data compression performance; and

refining, based on the data compression performance, the ANN evaluating.

10. The computer system of claim 6, wherein the evaluating using an ANN comprises:

determining a set of weighting factors describing the ANN;

applying, to the first training data stream input and the second training data stream input, the set of weighting factors; and

generating, by the applying, the first training data stream output and the second training data stream output.

11. A computer program product for handling data based on compressibility, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

identifying a first training data stream input, the first data stream input comprising a sample of fully compressed data;

identifying a second training data stream input, the second data stream input comprising a sample of compressible data;

evaluating, using an artificial neural network (ANN), the first training data stream input and the second training data stream input;

tagging, as fully compressed, the first training data stream output; and

tagging, as compressible, the second training data stream output.

12. The computer program product of claim 11, wherein the method further comprises:

identifying, following the evaluating of the first and second training data stream inputs, a digital data stream to be compressed by a data compressor module;

evaluating, using the ANN, the digital data stream to be compressed by the data compressor module;

determining, based on the evaluating, the data in the digital data stream is not further compressible; and

storing the fully compressed digital data in the digital data stream.

13. The computer program product of claim 11, wherein the method further comprises:

identifying, following the evaluating of the first and second training data stream inputs, a digital data stream to be compressed by a data compressor module;

evaluating, using the ANN, the digital data stream to be compressed by the data compressor module;

determining, based on the evaluating, a subset of data in the digital data stream is further compressible data; and

transmitting, to the data compressor module, the subset of data.

14. The computer program product of claim 11, wherein the method further comprises:

identifying, following the evaluating of the first and second training data stream inputs, digital data to be compressed by a data compressor module;

evaluating, using the ANN, the digital data to be compressed;

determining, based on the evaluating, a subset of the digital data is fully compressed;

transmitting, to the data compressor module and for compression, the subset of fully compressed data;

determining, based on the fully compressed data from the data compressor module, data compression performance; and

refining, based on the data compression performance, the ANN evaluating.

15. The computer program product of claim 11, wherein the evaluating using an ANN comprises:

determining a set of weighting factors describing the ANN;

applying, to the first training data stream input and the second training data stream input, the set of weighting factors; and

generating, by the applying, the first training data stream output and the second training data stream output.