ELECTRONIC APPARATUS FOR COMPRESSION AND DECOMPRESSION OF DATA AND COMPRESSION METHOD THEREOF

• US 20190348997A1
• Filed: 01/04/2019
• Published: 11/14/2019
• Est. Priority Date: 05/09/2018
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: LEE, Dongsoo, Cho, Youngchul, Son, Kwanghoon, Kim, Byeoungwook
• Granted Patent: US 10,608,664 B2
• Time in Patent Office: Days
• CPC Class Codes: G05B 13/00 Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion G05B19/00 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 7/02 using fuzzy logic computing arrangements based on biological models G06N3/00; computing arrangements using knowledge-based models G06N5/00
• CPC Class Codes: H03M 1/12 Analogue/digital converters H03M1/001 – H03M1/10 take precedence
• CPC Class Codes: H03M 13/41 using the Viterbi algorithm or Viterbi processors
• CPC Class Codes: H03M 13/4107 implementing add, compare, select [ACS] operations
• CPC Class Codes: H03M 13/6312 Error control coding in combination with data compression
• CPC Class Codes: H03M 5/00 Conversion of the form of the representation of individual digits
• CPC Class Codes: H03M 7/00 Conversion of a code where information is represented by a given sequence or number of digits to a code where the same , similar or subset of information is represented by a different sequence or number of digits
• CPC Class Codes: H03M 7/3024 having one quantiser only
• CPC Class Codes: H03M 7/3059 Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
• CPC Class Codes: H03M 7/3066 by means of a mask or a bit-map

Abstract

A data compression method and a data decompression method are provided. The method includes pruning an original data including a plurality of weight parameters, identifying at least one first weight parameter of which at least one first value is not changed by the pruning, among multiple weight parameters included in the pruned original data, and obtaining a first index data including location information of the at least one first weight parameter of which the at least one first value is not changed, identifying at least one second weight parameter of which at least one second value is changed by the pruning, among the multiple weight parameters included in the pruned original data, and substituting the at least one second weight parameter of which the at least one second value is changed with a don'"'"'t care parameter.

Claims

1. A data compression method of compressing a weight parameter of an artificial intelligence model, the method being performed by an electronic apparatus, and the method comprising:

pruning an original data comprising a plurality of weight parameters;

identifying at least one first weight parameter of which at least one first value is not changed by the pruning, among multiple weight parameters included in the pruned original data;

obtaining a first index data comprising location information of the at least one first weight parameter of which the at least one first value is not changed;

identifying at least one second weight parameter of which at least one second value is changed by the pruning, among the multiple weight parameters included in the pruned original data;

substituting the at least one second weight parameter of which the at least one second value is changed with a don'"'"'t care parameter;

quantizing, to a n bit, a first data comprising the at least one first weight parameter of which the at least one first value is not changed and the don'"'"'t care parameter with which the at least one second weight parameter is substituted;

obtaining a n number of second data, based on the first data quantized to the n bit; and

obtaining a n number of compressed data by applying, to a Viterbi algorithm, each of the obtained n number of second data.

2. The data compression method as claimed in claim 1, wherein the first data is in a matrix form, and

wherein the quantizing the first data comprises quantizing each of the at least one first weight parameter of which the at least one first value is not changed and the don'"'"'t care parameter with which the at least one second weight parameter is substituted, in the matrix form.

3. The data compression method as claimed in claim 2, wherein the n number of second data comprises:

a 2-1 data comprising a first bit value of the at least one first weight parameter quantized to the n bit, and a first bit of the don'"'"'t care parameter quantized to the n bit; and

a 2-2 data comprising a second bit value of the at least one first weight parameter quantized to the n bit, and a second bit of the don'"'"'t care parameter quantized to the n bit.

4. The data compression method as claimed in claim 1, wherein the obtaining the n number of compressed data comprises:

comparing an output data that is output by inputting at least one compression candidate data to at least one Viterbi decompressor, with the obtained n number of second data; and

based on the obtained n number of second data and the output data being matched at over a predetermined ratio, obtaining the at least one compression candidate data, as the n number of compressed data.

5. The data compression method as claimed in claim 4, wherein the comparing comprises comparing only pieces of data corresponding to the at least one first weight parameter of which the at least one first value is not changed, among the n number of second data, with the output data.

6. The data compression method as claimed in claim 1, further comprising obtaining a compressed index data by applying the first index data to the Viterbi algorithm.

7. The data compression method as claimed in claim 6, wherein the obtaining the compressed index data comprises:

comparing an output data that is output by inputting at least one compression candidate data to a Viterbi-decompressor, with the first index data; and

based on the first index data and the output data being matched at over a predetermined ratio, obtaining the at least one compression candidate data, as the compressed index data.

8. The data compression method as claimed in claim 6, further comprising storing the compressed index data, the n number of compressed data, and a table data for the quantizing.

9. An electronic apparatus for compressing a weight parameter of an artificial intelligence model, the apparatus comprising:

a memory storing instructions; and

a processor configured to execute the stored instructions to;

prune an original data comprising a plurality of weight parameters;

identify at least one first parameter of which at least one first value is not changed by the pruning, from among multiple weight parameters included in the pruned original data;

obtain a first index data comprising location information of the at least one first weight parameter of which the at least one first value is not changed;

identify at least one second weight parameter of which at least one second value is changed by the pruning, among the multiple weight parameters included in the pruned original data;

substitute the at least one second weight parameter of which the at least one second value is changed with a don'"'"'t care parameter;

quantize, to a n bit, a first data comprising the at least one first weight parameter of which the at least one first value is not changed and the don'"'"'t care parameter with which the at least one second weight parameter is substituted;

obtain a n number of second data, based on the first data quantized to the n bit; and

obtain a n number of compressed data by applying, to a Viterbi algorithm, each of the obtained n number of second data.

10. The electronic apparatus as claimed in claim 9, wherein the first data is in a matrix form, and

wherein the processor is further configured to execute the stored instructions to quantize each of the at least one first weight parameter of which the at least one first value is not changed and the don'"'"'t care parameter with which the at least one second weight parameter is substituted, in the matrix form.

11. The electronic apparatus as claimed in claim 10, wherein the n number of second data comprises:

a 2-1 data comprising a first bit value of the at least one first weight parameter quantized to the n bit, and a first bit of the don'"'"'t care parameter quantized to the n bit; and

a 2-2 data comprising a second bit value of the at least one first weight parameter quantized to the n bit, and a second bit of the don'"'"'t care parameter quantized to the n bit.

12. The electronic apparatus as claimed in claim 9, wherein the processor is further configured to execute the stored instructions to:

compare an output data that is output by inputting at least one compression candidate data to at least one Viterbi decompressor, with the obtained n number of second data; and

based on the obtained n number of second data and the output data being matched at over a predetermined ratio, obtain the at least one compression candidate data, as the n number of compressed data.

13. The electronic apparatus as claimed in claim 12, wherein the processor is further configured to execute the stored instructions to compare only pieces of data corresponding to the at least one first weight parameter of which the at least one first value is not changed, among the n number of second data, with the output data.

14. The electronic apparatus as claimed in claim 9, wherein the processor is further configured to execute the stored instructions to obtain a compressed index data by applying the first index data to the Viterbi algorithm.

15. The electronic apparatus as claimed in claim 14, wherein the processor is further configured to execute the stored instructions to:

compare an output data that is output by inputting at least one compression candidate data to a Viterbi-decompressor, with the first index data; and

based on the first index data and the output data being matched at over a predetermined ratio, obtain the at least one compression candidate data, as the compressed index data.

16. The electronic apparatus as claimed in claim 14, wherein the memory is further configured to store the compressed index data, the n number of compressed data, and a table data for quantization of the first data.

17. A decompression method of decompressing a compressed data, the method being performed by an electronic apparatus, and the method comprising:

receiving a compressed index data and a n number of compressed data;

obtaining a first index data by applying the compressed index data to a first Viterbi decompressor;

obtaining a n number of first data by applying the n number of compressed data respectively to a plurality of Viterbi decompressors; and

obtaining a pruned original data, based on the first index data and the n number of first data.

18. The decompression method as claimed in claim 17, further comprising:

inputting the pruned original data to an artificial intelligence model;

retraining the artificial intelligence model to which the pruned original data is input; and

obtaining a plurality of weight parameters applied to the artificial intelligence model, based on a result of the retraining.

19. An electronic apparatus for decompressing a compressed data, the apparatus comprising:

a memory storing instructions; and

a processor configured to execute the stored instructions to;

receive a compressed index data and a n number of compressed data;

obtain a first index data by applying the compressed index data to a first Viterbi decompressor;

obtain a n number of first data by applying the n number of compressed data respectively to a plurality of Viterbi decompressors; and

obtain a pruned original data, based on the first index data and the n number of first data.

20. The electronic apparatus as claimed in claim 19, wherein the processor is configured to:

input the pruned original data to an artificial intelligence model;

retrain the artificial intelligence model to which the pruned original data is input; and

obtain a plurality of weight parameters applied to the artificial intelligence model, based on a result of retraining the artificial intelligence model.

21. A data compression method being performed by an electronic apparatus, the method comprising:

pruning an original data comprising a plurality of weight parameters of an artificial intelligence model;

obtaining a first index data comprising location information of at least one first weight parameter of which at least one first value is not changed by the pruning, among multiple weight parameters included in the pruned original data;

obtaining a first data by;

substituting, with a don'"'"'t care parameter, at least one second weight parameter of which at least one second value is changed by the pruning, among the multiple weight parameters included in the pruned original data; and

quantizing, to a n bit, the at least one first weight parameter of which the at least one first value is not changed and the don'"'"'t care parameter with which the at least one second weight parameter is substituted;

obtaining a second data, based on the first data quantized to the n bit; and

obtaining a compressed index data and a compressed data by applying, to a Viterbi algorithm, each of the first index data and the second data.

22. The data compression method of claim 21, wherein the second data comprises:

a 2-1 data comprising a first bit value of the at least one first weight parameter quantized to the n bit, and a first bit of the don'"'"'t care parameter quantized to the n bit; and

a 2-2 data comprising a second bit value of the at least one first weight parameter quantized to the n bit, and a second bit of the don'"'"'t care parameter quantized to the n bit.