Data Compression/Decompression Device

• US 20150100556A1
• Filed: 04/18/2013
• Published: 04/09/2015
• Est. Priority Date: 05/25/2012
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Nagai, Yasushi, Osafune, Tatsuaki, Sekiguchi, Takaaki, Fukunaga, Ryoichi, Okubo, Takahiro, Imai, Taiki
• Granted Patent: US 10,116,325 B2
• Time in Patent Office: Days
• US Class Current: 707/693
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/0864 using pseudo-associative means, e.g. set-associative or hashing
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 12/0895 of parts of caches, e.g. directory or tag array
• CPC Class Codes: G06F 12/126 with special data handling, e.g. priority of data or instructions, handling errors or pinning
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/465 Structured object, e.g. database record
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
• CPC Class Codes: H03M 7/40 Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
• CPC Class Codes: H03M 7/46 Conversion to or from run-length codes, i.e. by representing the number of consecutive digits, or groups of digits, of the same kind by a code word and a digit indicative of that kind

Abstract

When compressing an arrangement of fixed-length records in a columnar direction, a data compression device carries out data compression aligned with the performance of a data decompression device by computing a number of rows processed with one columnar compression from the performance on the decompression device side, such as the memory cache capacity of the decompression device or the capacity of a primary storage device which may be used by an application, and the size of one record. Thus, while improving compression ratios of large volumes of data, including an alignment of a plurality of fixed-length records, decompression performance is improved.

Claims

1. A data decompression device that decompresses compressed data including a plurality of compressed fixed-length records, wherein:

the compressed data includes a result acquired by dividing compression object data in units of predetermined compression object block size and compressing the data every compression object block;

when the compression object block size is determined based upon the size of the fixed-length record and the specification information of the data decompression device, the size of the compression object block is determined based upon the size of the fixed-length record and the specification information of the data decompression device;

one or more compression object blocks are acquired based upon the compressed data; and

a columnar data decompression unit that decompresses each compressed columnar data as a result of compressing the same columnar data of the plurality of fixed-length records included in the compression object block every column and restores the plurality of fixed-length records is provided.

2. The data decompression device according to claim 1, further comprising:

a software execution unit that executes software, wherein;

the compressed data includes one or more blocks each of which is a result of compression every compression object block and each of which includes one or more subblocks; and

a block data decompression unit that controls the decompression of each of one or more blocks including data requested by the software is included.

3. The data decompression device according to claim 1, wherein:

the specification information of the data decompression device is the capacity of a cache memory with which the data decompression device is provided; and

the size of the compression object block is a maximum multiple of the size of the fixed-length record which does not exceed a value acquired by multiplying the capacity of the cache memory by predetermined ratio.

4. The data decompression device according to claim 3, wherein:

the predetermined ratio by which the capacity of the cache memory is multiplied is 0.5.

5. The data decompression device according to claim 1, wherein:

the specification information of the data decompression device is the capacity of a main storage which the software secures to hold decompressed data of the compression object block; and

the size of the compression object block is a maximum multiple of the size of the fixed-length record which does not exceed the capacity of the main storage.

6. The data decompression device according to claim 1, wherein:

the specification information of the data decompression device is maximum size of data which can be transmitted once by a communication device provided to the data decompression device; and

the size of the compression object block is a maximum multiple of the size of the fixed-length record which does not exceed the maximum size of the data which can be transmitted once.

7. The data decompression device according to claim 1, comprising:

a compressed data request unit that transmits a name of compressed data and the specification information of the data decompression device to a data compression device via a communication network and receives the compressed data suitable for the notified information from the data compression device.

8. The data decompression device according to claim 7, wherein:

the compressed data request unit displays a warning screen showing that no suitable compressed data exists when it is notified from the data compression device that no suitable compressed data exists.

9. A data compression device that compresses compression object data including a plurality of fixed-length records, comprising:

a unit size setting unit that accepts the input of the size of one fixed-length record and the specification information of a data decompression device; and

a columnar data compression unit that determines the size of a compression object block based upon the size of the fixed-length record and the specification information of the data decompression device, compresses the same columnar data of the plurality of fixed-length records included in each compression object block every column, generates compressed columnar data and generates compressed data including each compressed columnar data.

10. The data compression device according to claim 9, wherein:

the specification information of the data decompression device is the capacity of a cache memory provided to the data decompression device; and

the size of the compression object block is a maximum multiple of the size of the fixed-length record which does not exceed a value acquired by multiplying the capacity of the cache memory of the decompression device by predetermined ratio.

11. The data compression device according to claim 10, wherein:

the predetermined ratio by which the capacity of the cache memory of the decompression device is multiplied is 0.5.

12. The data compression device according to claim 9, comprising:

a block data compression unit that divides the compression object data into blocks of predetermined size and controls the compression of each block, wherein;

the block data compression unit determines a method of compressing each block referring to a compression definition file in which record length different in each compression object data of a fixed-length record is described; and

the columnar data compression unit compresses the block by columnar compression determined as a compression method by the block data compression unit as compression object data.

13. The data compression device according to claim 9, wherein:

a plurality of compressed data which the columnar data compression unit generates from one compression object data based upon a plurality of specification information input to the unit size setting unit are stored in a storage medium, further comprising;

a compressed data selection unit that receives the specification information of the data decompression device from the data decompression device via a communication network, selects compressed data suitable for the received specification information, and transmits the compressed data to the data decompression device.

14. The data compression device according to claim 9, further comprising:

a decompression unit that decompresses compressed data when the specification information of the data decompression device received from the data decompression device via a communication network is different from the specification information referred in generating the compressed data, wherein;

one compressed data generated based upon one compression object data is stored in the storage medium; and

recompressed compressed data based upon the specification information of the data decompression device received from the data decompression device using decompressed data output from the decompression unit for compression object data is transmitted to the data decompression device.

15. A program for making a computer function as an information processor that decompresses compressed data including a plurality of compressed fixed-length records, wherein:

the compressed data includes a result acquired by dividing compression object data in units of predetermined compression object block size and compressing every compression object block; and

when the compression object block size is determined based upon the size of the fixed-length record and the specification information of the information processor, the program makes the computer execute steps of;

determining the size of the compression object block based upon the size of the fixed-length record and the specification information of the information processor;

acquiring one or more compression object blocks based upon the compressed data; and

decompressing each compressed columnar data which is a result of compressing the same columnar data of the plurality of fixed-length records included in the compression object block every column and restoring the plurality of fixed-length records.

16. A program for making a computer function as an information processor that compresses compression object data including a plurality of fixed-length records, wherein:

the computer is provided with an interface that accepts an instruction from a user; and

the program makes the computer execute steps of;

accepting the input of the size of one fixed-length record and the specification information of a data decompression device via the interface;

determining the size of a compression object block based upon the size of the fixed-length record and the specification information of the data decompression device;

dividing the compression object data into one or more compression object blocks;

compressing the same columnar data of the plurality of fixed-length records included in each compression object block every column and generating compressed columnar data; and

generating compressed data including each compressed columnar data.

17. A compressed data distribution system in which a data compression device and a data decompression device are connected via a communication network, wherein:

the data compression device is provided with;

a unit size setting unit that accepts the input of the size of one fixed-length record and the specification information of the data decompression device; and

a columnar data compression unit that determines the size of a compression object block based upon the size of the fixed-length record and the specification information of the data decompression device, divides compression object data into one or more compression object blocks, compresses the same columnar data of the plurality of fixed-length records included in each compression object block every column, generates compressed columnar data and generates compressed data including each compressed columnar data;

the compressed data includes a result acquired by dividing the compression object data in units of predetermined compression object block size and compressing every compression object block; and

the data decompression device is provided with a columnar data decompression unit which determines the size of the compression object block based upon the size of the fixed-length record and the specification information of the data decompression device when the compression object block size is determined based upon the size of the fixed-length record and the specification information of the data decompression device, which acquires one or more compression object blocks based upon the compressed data, which decompresses each compressed columnar data as a result of compressing the same columnar data of the plurality of fixed-length records included in the compression object block every column and which restores the plurality of fixed-length records.