System and method for data compression using multiple small encoding tables

• US 8,872,679 B1
• Filed: 12/05/2012
• Issued: 10/28/2014
• Est. Priority Date: 12/05/2012
• Status: Active Grant

• Current Assignee: Teradata Corporation
• Original Assignee: Teradata US, Inc. (Teradata Corporation)
• Inventors: Roberts, Gary, Wang, Guilian
• Primary Examiner(s): YOUNG, BRIAN K
• Application Number:
  US13/705,217
• Time in Patent Office: 692 Days
• Field of Search: 341/51, 341/65, 341/67, 341/87, 341/106, 341/107
• US Class Current: 341/106
• CPC Class Codes: H03M 7/42 using table look-up for the coding or decoding process, e.g. using read-only memory H03M7/4006 takes precedence
• CPC Class Codes: H03M 7/607 Selection between different types of compressors
• CPC Class Codes: H03M 7/705 Unicode

Abstract

A system and method for compressing and decompressing multiple types of character data. The system and method employ multiple encoding tables, each designed for encoding a subset of character data, such as numeric data, uppercase letters, lowercase letters, Latin, or UNICODE data, to perform compressions and decompression of character data, and. The compression solution also provides for the creation of new encoding tables on the fly, and accommodates the compression of lengthy character streams using multiple different general compression algorithms, automatically choosing a suitable general compression algorithm for specific input data.

Claims

1. A method for compressing character data, the method comprising the steps of:

maintaining within a computer system a plurality of encoding tables corresponding to a plurality of character alphabets;

receiving, by said computing system, an input stream containing uncompressed character data;

determining, by said computing system, the length of said input string containing uncompressed character data;

when the length of said input stream exceeds a predefined length value, compressing, by said computing system, said uncompressed character data using a general purpose compression algorithm; and

when the length of said input stream does not exceed said predefined length value, selecting, by said computing system, an encoding table from said plurality of encoding tables, said selected encoding table corresponding to the character alphabet of said uncompressed character data contained within said input stream, and compressing, by said computing system, said uncompressed character data using said selected encoding table to provide compressed character data.

2. The method for compressing data in accordance with claim 1, wherein said general purpose compression algorithm is selected from the following general purpose compression algorithms:

Uniform Transformation Format 8 (UTF8);

Standard Compression Scheme for Unicode (SCSU);

Lempel-Ziv Compression Scheme (ZLIB);

Huffman Run Length Encoding; and

Binary Ordered Compression for Unicode (BOCU).

3. The method for compressing character data in accordance with claim 1, further comprising the step of:

when the length of said input stream does not exceed said predefined length value, and said plurality of encoding table does not include an encoding table corresponding to the character alphabet of said uncompressed character data, analyzing, by said computing system, said input string to identify a new character alphabet and creating a new encoding table to compress said new character alphabet.

4. A system for compressing data, comprising:

a database management system including a compression service that;

receives an input stream containing uncompressed character data;

determines the length of said input string containing uncompressed character data;

when the length of said input stream exceeds a predefined length value, compresses said uncompressed character data using a general purpose compression algorithm; and

when the length of said input stream does not exceed said predefined length value, selects an encoding table from a plurality of encoding tables corresponding to a plurality of character alphabets, said selected encoding table corresponding to the character alphabet of said uncompressed character data contained within said input stream, and compresses said uncompressed character data using said selected encoding table to provide compressed character data.

5. The system for compressing data in accordance with claim 4, wherein said general purpose compression algorithm is selected from the following general purpose compression algorithms:

Uniform Transformation Format 8 (UTF8);

Standard Compression Scheme for Unicode (SCSU);

Lempel-Ziv Compression Scheme (ZLIB);

Huffman Run Length Encoding; and

Binary Ordered Compression for Unicode (BOCU).

6. The system for compressing data in accordance with claim 4, wherein:

when the length of said input stream does not exceed said predefined length value, and said plurality of encoding table does not include an encoding table corresponding to the character alphabet of said uncompressed character data, said compression service analyzes said input string to identify a new character alphabet and creates a new encoding table to compress said new character alphabet.

7. A non-transitory computer-readable medium having a computer program, for compressing an input string containing uncompressed character data having one of a plurality of alphabets received by a computer system, the computer program including executable instructions that cause said computer system to:

determine the length of said input string;

when the length of said input stream exceeds a predefined length value, compress said uncompressed character data using a general purpose compression algorithm; and

when the length of said input stream does not exceed said predefined length value, select an encoding table from a plurality of encoding tables, said selected encoding table corresponding to the character alphabet of said uncompressed character data contained within said input stream, and compress said uncompressed character data using said selected encoding table to provide compressed character data.

8. The non-transitory computer-readable medium having a computer program for compressing character data having one of a plurality of alphabets received by a computer system in accordance with claim 7, wherein said general purpose compression algorithm is selected from the following general purpose compression algorithms:

Uniform Transformation Format 8 (UTF8);

Standard Compression Scheme for Unicode (SCSU);

Lempel-Ziv Compression Scheme (ZLIB);

Huffman Run Length Encoding; and

Binary Ordered Compression for Unicode (BOCU).

9. The non-transitory computer-readable medium having a computer program for compressing character data having one of a plurality of alphabets received by a computer system in accordance with claim 7, wherein:

when the length of said input stream does not exceed said predefined length value, and said plurality of encoding table does not include an encoding table corresponding to the character alphabet of said uncompressed character data, the program includes executable instructions that cause said computer system to analyze said input string to identify a new character alphabet and create a new encoding table to compress said new character alphabet.